 he Importance Of Certification
Certification is a tool that objectively measures the performance of a
professional on a certain job at a defined skill level. Certification is
therefore beneficial for developers who wish to validate their own skills as
well as for companies who wish to validate the skills of their employees.
Whether or not certification is an effective measurement tool depends on
what basis the certification tests are developed. For optimum benefit, the
certification tests must be developed to truly reflect the critical tasks
required for a job, the skill levels of each task, and the frequency by which a
task needs to be performed.
This valid and defensible development approach was used to create the
product-specific Object Technology Certifications which focus on VisualAge
for Java, VisualAge for C++, VisualAge for COBOL and VisualAge
for Smalltalk.
In addition to assessing job skills and performance levels, object
technology certification can also provide benefits such as:
For the employers:
- Measures the effectiveness of training
- Reduces course redundancy and unnecessary expenses
- Provides objective benchmarks to help determine raises and promotions
- Makes long-range planning easier
- Helps manage professional development
- Aids as a hiring tool
- Contributes to competitive advantage
- Increases productivity
- Increases morale and loyalty
For the employee:
- Provides industry recognition as an IBM Object Technology Certified
professional
- Validates your skills
- Provides advantages in a job interview
- Helps in job location and retention
- Increases salary
- Increases self esteem
- Provides continuing professional benefits
The Certification Test Development Process
The Object Technology Certification process began with a market need -
professional service providers who can create object-oriented applications using
either VisualAge for Java, VisualAge for C++, VisualAge for COBOL or VisualAge
for Smalltalk at the same desired competency level. For each product, a
thorough study was conducted with a small number of subject matter experts and a
larger representative population of the service provider job role. The studies
focused on the tasks required for the job role, the skill level at which they
were required, and how frequently they needed to be performed. The analysis of
each study served as the foundation of all subsequent test development
activities.
From the analysis of each study, effective blueprints for the tests were
created. The blueprints determined how the tests were to be proportioned on a
descending order of the 'critical' required tasks and their associated skill
levels. The blueprints also determined the test objectives, which served as a
guide for identifying, building, and modifying educational offerings that
supported the certification tests. Lastly, the blueprints served as a guide for
selecting the best assessment methods. In the case of the Object Technology
Certifications, both knowledge-based and performance-based testing were
selected. Initially, a certification candidate's knowledge is tested through
multiple one to one and a half hour, closed-book, multiple-choice, computerized
tests of knowledge that the candidate may have obtained from on-the-job
training, reading publications, or attending courses. Subsequently, the
candidate's skills are tested via a three and a half or five day
performance-based test. Here the candidate must apply the knowledge, by
prototyping in a team environment, to develop a solution to a case study that
simulates an actual application engagement using either VisualAge for Java,
VisualAge for C++, VisualAge for COBOL, or VisualAge for
Smalltalk.
Once the test objectives were created and the educational needs and
assessment methods were identified, under the guidance of a testing
professional, experts with related practical experience created realistic
knowledge-based questions (many based on scenarios) and performance-based case
studies that linked directly to and supported the test objectives.
A new set of experts then reviewed the content of each knowlege-based test
question or performance-based case study against the job role. They also
determined the "cut-off" or required score a candidate would have to
achieve in order to pass a particular test. The "cut-off" score is the
score that draws the line between those candidates who successfully complete a
test and those who do not. This score is determined by professional input.
Subject matter experts define what the borderline or minimally competent
certification candidate would need to know.
For the knowledge-based test, the experts assigned a probability level to
each test question, which is the probability whether a well-prepared test
candidate would or would not answer each test question correctly. From this
data, a final "cut-off" score was established.
For the performance-based tests, categories were defined, each containing a
number of rating elements. Observable skills levels were also identified for
each rating element. Individuals who would rate candidates' individual
performances during the performance-based test via a predefined score against
the observable skills were trained to ensure inter-rater reliability.
Inter-rate reliability ensures that an individual's performance rating by one
rater or assessor will match that of another rater.
Pilots of each test were conducted with service providers that are
representative of our target audience.
After a designated number of candidates completed a knowledge-based test
during each pilot, a test question analysis involving examination of the pattern
of multiple-choice responses was conducted on the knowledge-based tests. Item
analysis is conducted to improve a test by assessing which questions of the test
need refinement and which should remain or be discarded. This can be determined
by identifying the discrimination level of a test question (e.g., which
multiple-choice option is never selected), item level difficulty (e.g., which
correct multiple-choice option is never, rarely, or too frequently selected) and
reliability (i.e., the degree by which the test consistently measures whatever
it intends to measure). After each pilot, adjustments to the tests were made as
deemed required with regards to clarity, technical accuracy, and relevance. The
product-specific Object Technology certification tests proved reliable. This
means that if the same group of people were to take the test twice, their scores
from the second attempt would closely match those of the first attempt.
Likewise, after a designated number of candidates completed a
performance-based test, an analysis was done on all aspects of the case study,
the rating elements, observable skills, and rater training. Following the
analysis, adjustments were made where needed.
Once the Object Technology certification tests were made "generally
available" worldwide through the Professional Certification Program from
IBM, all data and aspects of the certification tests are carefully tracked and
monitored. The product-specific object technology certification tests are
continuously reviewed and updated to ensure that the required knowledge and
skills for an object-oriented, application development service provider are
current.
Conclusion
The object technology certification tests focusing on VisualAge for Java,
VisualAge for C++, VisualAge for COBOL and VisualAge for
Smalltalk are created based on a particular job role, the tasks needed for
that job, the skill levels needed for each of the tasks, and the frequency at
which the task needs to be performed. Because of this development approach,
object technology certification is an effective tool that objectively measures
the performance of a professional delivering object-oriented, application
development services.
For more information on how you or your associates/employees, can become IBM
Object Technology Certified Professionals, visit the IBM Object Technology
Certification web site at http://www.software.ibm.com/ad/certify/ot | 
