Discussion 5

Happy March everybody! (okay, not quite yet) Hopefully everyone is a little less stressed out now that the holidays are behind us. Now if I can just convince management to pick up some of those SROA folk out there to do some job analysis...
 
*** New Member ***
 
Our group just keeps getting bigger! Please welcome Lisa Kehoe from the Managed Risk Medical Insurance Board (LKehoe@mrmib.ca.gov). Lisa, we look forward to your contributions!
 
*** News ***
 
- The big news this month is that at it's February 3rd meeting, the SPB approved the Decentralized Testing Program final report. In addition, the SPB adopted Article 3.5, Section 50 in Title 2 of the CAR which incorporates by reference SPB's Merit Selection Manual. You may have seen a pinkie come out about both of these issues (available at P030227.pdf).
 
This is huge news for CA civil service and testing in particular. Are there implications for job analysis? You bet. Let's take a look at what SPB has to say about job analysis:
 
1) In the Summary of Findings and Discussion (p. 104), SPB notes, "The most serious deficiency identified is the lack of job-relatedness of many of the testing processes. A large majority of decentralized testing processes are not based on job analyses which meet the job analytic requirements set forth by the Uniform Guidelines, nor do these processes demonstrate evidence of validity (i.e., documented evidence of a relationship between test performance and job performance)."
 
2) On page 1 of Appendix L, "Standards for decentralized departmental examination programs", they note that exams "shall" include several factors, including:
 
* The use of soundly developed, job-related examination processes and individual selection procedures, developed based upon job analytic data...
* The use of selection procedures appropriate to assess those KSAs identified as important for successful job performance and required-upon-entry to the job.
* Documentation linking the content of the examination process and its individual selection procedures to the content requirements of the job classification...
 
3) On the very last page of Appendix L, they list as duties for Examination Analysts (journey-level):
 
* Conduct job analysis studies of varying degrees of complexity utilizing recognized, legally defensible job analysis methodologies
* Design and develop job-related, content-valid, legally defensible examination processes and individual selection procedures
* Document the content validity of examination processes and individual selection procedures...
 
- The Selection Manual also includes important references to job analysis, including the chart in Section 1200 titled "Conducting a Civil Service Examination Process". You'll notice that "conduct job analysis/collect job analytic data" is the first item listed under exam design and planning activities.
 
Bottom line: I'm guessing that departments will increasingly be held accountable for the job-relatedness of their exams. The best way to achieve this? You get one guess.
 
*** Proposal ***
 
Since there was no argument with my proposal last time to start a list of job analyses that have been completed, I will go ahead and begin the process. I encourage you all to let me know what analyses you have completed so we can SHARE THE KNOWLEDGE. I will put the first foot forward and tell you we have finished two so far, Criminal Intelligence Specialist I and Special Agent Supervisor. I am currently working on Deputy Attorney General, Special Agent, and Special Agent Trainee.
 
*** Resources ***
 
This month's resource is a biggie. Many of you I'm sure have conducted job analysis only to say something to the effect of, "Okay, the SMEs identified Creativity. That's great. What am I supposed to use to test that?" Well, luckily there is a whole industry out there that just loves to create exams (can you imagine?). I ran across a chart on-line that may help us all. I'm not suggesting we all drop our exam development, but you know what they say about reinventing the wheel. I have attached my current revision of this chart as a MS Word file; I'm still tinkering with it, but I hope it will help some of you out there. As always, if you have any questions about purchasing off-the-shelf exams, please ask the group!
 
***
 
In closing, I'd like to share a thought I had. I'm currently conducting a very large job analysis that will most likely take the better part of this year. I was bemoaning this (which is saying a lot, given how big of a job analysis nut I am) until I realized that, if done right, we wouldn't have to do it again for a long time (at least not a full-blown one). This made me think of a t-shirt that barbri, a Bar exam prep company, hands out to its students. On the back it says: "Do it once. Do it right. Never do it again." Maybe SPB can produce shirts that say "Job analysis" on the front, and on the back: "Do it once. Do it right. Never do it again (or at least not until the job changes significantly enough to warrant a new one)."
 
Peace-
 
Bryan Baldwin