Discussion 14

Good Morning Everyone,

I was wondering, when gathering job analysis information do you specify a
performance level for SMEs? Typically, when conducting the job analysis and
subsequent test development I ask for top performers because I fell that
they generally have a better understanding of the job, particularly in
regards to how it relates to overall organizational performance. However, I
was thinking that it could be possible that "stars" tend to engage in extra
role tasks and behaviors that go beyond the scope of their job (i.e., OCBs).
Therefore, it is quite possible that these stars are describing the job of a
star rather than the job at a basic competency level. I have not found
anything in the literature related to the skill level of job analysis SMEs,
so I thought why not ask for advice from the list-serv. Thanks.

Juan Benavidez
Human Resource Management Specialist
State of Oklahoma
Office of Personnel Management
Personnel Assessment Division
Jim Thorpe Building
2101 N. Lincoln Blvd
Oklahoma City, OK 73105-4904
(405) 522-1201
juan.benavidez@opm.state.ok.us 

OPM Mission: " We serve the people of Oklahoma by delivering reliable and
innovative human resource services to our partner agencies to achieve their
missions".

---

Good question . . . .at DHS, we request experienced SMEs, and do not distinguish them by their competency level. When rating the Tasks/KSAs we canvas all incumbents and their supervisors to ensure the broadest range of responses. You make a good point about the "stars" sometimes having higher standards. We emphasize "first day on the job" with these SMEs and ask continuously "did you do this your first day on the job or did you learn to do it while on the job?" This helps keep them on track.

Barbara Warner, Chief
Selection Unit
1501 Capitol Avenue, Suite 71.1501
MS 1301
(916) 552-8336 / FAX (916) 552-8275
mailto:bwarner@dhs.ca.gov 

---

Good Morning Juan,
 
At Dept of Mental Health, we ask for SMEs who have the experience/technicial knowledge in the class, not specifically, the top performers. Similar to DHS, we also emphasize with the SMEs, that for examining purposes, we are focusing on the KSAs required before appointment to the class (at the point of entry) rather than those learned on the job.
 
Regena Caton, Manager
Selection Services
(916) 654-3764

---

Juan - Our approach at DOJ is similar to the responses that Barbara and
Regena gave. We make it clear that the overriding consideration for the
selection of SMEs is their knowledge of the job and what it takes to get
the job done. We often use one year of experience as a qualification.
It is sometimes necessary to re-focus SMEs on entry-level requirements,
but most tend to stay on target.

To some extent the selection of SMEs depends on the classification, the
type of job analysis, and operational considerations. If we're doing a
job analysis that involves in-depth interviews and SME input, other
criteria (e.g., good communication skills, good teamwork ability) may be
as important, if not more important, than performance level in selecting
SMEs. If the classification has a small number of incumbents, or the
incumbents are very difficult to obtain, we may be forced to take
whomever we can get (of course this puts additional burden on the
analyst to make sure that the analysis is conducted properly and
thoroughly).

In general I think the best practice is to obtain input from a variety
of individuals, focusing on diversity in terms of demographics as well
as job duties, location, etc. The literature I am aware of (e.g., Landy
& Vasey, 1991) support this approach, particularly as it pertains to
experience level. I try to always keep in mind that selection of SMEs
has both political (e.g., fairness) as well as practical (e.g.,
reliability) implications.

Quickly reviewing the articles I have, the only study that looks
directly on point is one that I don't have: Henry & Morris (2000, April)
Incumbent performance level as a predictor of JA ratings. Paper
presented at the 15th Annual SIOP Convention. Scott Morris
(http://www.iit.edu/~morris/vita.html) may be able to provide you with a
copy--if so, let us know so we can all benefit!

Bryan Baldwin
Staff Services Manager I
California Dept. of Justice
(916) 322-0934