Discussion 9

Hello everyone! It's been quite a while since my last e-mail, and my "content" file has grown to the point where I thought it time for another update.

** News **

1. First and foremost, due to the diligent efforts of Karl Jaeger and others at SPB, these discussions are now available on the web! To access them, go to TV&C's webpage at www.spb.ca.gov/tvc and click on "Job Analysis Discussion List Archives" located on the left-hand side. Thanks to Karl and SPB--great job!

2. The Federal Office of Management and Budget (OMB) has, for the eighth time since the end of 2001, approved a three-month extension of the Uniform Guidelines on Employee Selection Procedures (UGESP). According to the OMB website, the current deadline for the UGESP is now December 31, 2003. Shocking, shocking news! (yes, that's sarcasm)

** Information **

1. As many of you know, the same types of KSAs often come up when conducting a job analysis. Written communication skill, oral communication skill, analytical ability, ability to get along with others, etc.--these KSAs apply to many jobs and are often identified by SMEs as important. The challenge, as always, is figuring out how to test for them. I started thinking about a way to summarize acceptable alternatives for measuring these common KSAs. And yes, I've come up with yet another Excel spreadsheet. It is attached, and is obviously a work in progress. If anyone has anything similar, or if you'd like to improve upon this one, please share the wealth!

2. The 27th Annual International Public Management Association Assessment Council (IPMAAC) Conference was held last June in Baltimore. There were many interesting-sounding presentation (I wish I could have gone!), but many of them are now available at this website: http://www.ipmaac.org/conf03/. If you browse through them, I'm sure you'll find something that tickles your fancy.

3. Speaking of IPMAAC, they have some great files on their website. One of them is a job analysis report that was done back in '95 on the "Personnel Assessment Specialist". It has some great task and KSA statements, linkages, and questionnaires included and could be used as part of your own analysis of, say, the APA classification. It is available at http://www.ipmaac.org/files/psjoban.pdf 

4. What part of job analysis is the most burdensome for SMEs? Did somebody say "linkages"? Remember, you do not HAVE to have the SMEs provide a numerical linkage value for each relationship. Instead, you can simply ask them to link task to KSAs that are at least "important" for performing that task. Now, most of you are probably thinking, "But Bryan, how do you calculate the weighted linkage values for each KSA?" (and if you are, 10 points). Simple: You just sum the IT and IK values for each task-KSA combination (e.g., treat the NP value as "1" on page 52 of the WRIPAC manual). Easier for them, easier for you.

** Issues **

1. Concept A or Staff Memo? I've heard it said in some circles that after conducting a job analysis, the testing analyst will prepare a "Concept A" paper describing the desired changes to the class specification (typically given to the C&P analyst). Recently I had occasion to review the C&P manual (okay, I had insomnia) and noticed that it quite clearly states that changes to knowledges/abilities resulting from a validation study require only a staff memo. Does anyone have experience going from job analysis to class spec revision? Which route did you go--staff or board item? Would it differ for a statewide classification?

2. On a similar note, what KSAs are you finding come up most often? In my experience, one of the top is writing skills. What have been your experience measuring this skill? We have recently utilized outside raters for both open-ended and problem-solving tasks, but I'm interested in hearing if anyone has done something different, perhaps utilized a commercial writing test?

** Misc **

Just thought I'd share something. We tried our first videoconferenced QAP two weeks ago and also did a panel orientation via videoconference. Both turned out great, but assistance from our tech support staff was essential. We hope this will save us considerable time and money in the future. Has anyone tried this before?

That's all for now! Take care and remember to breathe!

Bryan Baldwin
Dept. of Justice

---

Hey Bryan,
 
Thanks for keeping us up to date on the job analysis info.
 
We, at DMH, have done videoconferencing for years. For the most part, it has been successful, although face-to-face interviews are preferred. You are right in that Tech support is essential. In the past, we've used it in situations when there are one or two candidates in another location, for example, southern California, when the majority of candidates are in Sacto) and it was cheaper than sending a panel down. Now that we're doing more statewide exams, and trying to significantly reduce travel where possible, due to budget constraints, we're finding that videoconferencing in some cases may cost us more in the long run. Recently we had a long running exam, 12 days, and conducted it by videoconferencing. We had some issues arise, namely we had to bump and take up the videoconferencing rooms for 12 days, but the other issue that we've begun to explore is the cost effectiveness.
 
From that experience, we have decided it necessary to evaluate the cost effectiveness of videoconferencing vs. traveling panels. I just put in a request for travel rather than videoconferencing for an exam we're in process of administering. Our figures estimate that is costs about $80.00 per hour/per site to videoconference, so in some cases, it is probably more cost efficient to travel, depending on the number & location of the panel members and competitors, the number of sites needed and the number of hours of interviews. Since we could have as many as 4-5 sites (3-4 panel members/different locations plus candidate in a different location), it could be costly.
 
All in all, we've had good experiences with videoconferencing and would recommend it in certain situations.

Regena Caton

---

KSAs & test types