Tag Archives: Assessment

Photographs As Items for Assessment – Free Example

Photographs of people in activity is a promising newer area for development of business-relevant assessments that has been in use for years in healthcare.  Originally developed in the Netherlands to help patients suffering from fear of pain when moving the body (kinesiophobia), the University of Maastricht’s website has details on citations and free compressed (zipped file) short version of the main test.

Clearly, this same approach could be used to develop more engaging employee and organizational assessments that may be difficult to fake, have better face validity, and more workplace fidelity than other types of items.  Further, with cheap and even free video sites, video items could also play a bigger role in future assessments.

Consider these possible fruitful examples.

a) Vocational Interest Assessment

Vocational interest tests help people identify career paths for which their interests, values, and aptitudes are particularly suited.  But most all are purely text-based.  What if each career alternative had photographs of the tasks in each job or job family, with video vignettes of major tasks?  Perhaps this could be a fun way to assess what activities and careers would ultimately help the person realize their goals.  Take another look at the picture at the top of this article.  It’s an actual picture from PHODA’s assessment, but couldn’t it represent the task of lifting articles out of a trunk for the job of a taxicab driver?

b) Employee Selection

Cognitive and knowledge-based tests are often used to select new employees, but not nearly as often or instead of the ubiquitous job interview.  What if good instruments could be developed, perhaps with a combination of item types, to include pictures?  People could rate pictures like this one on the degree to which it looks similar to their desk – would you expect highly conscientious people to endorse this picture?

I would guess that highly conscientious and prudent people would be unlikely to indicate that this picture reflects their own office.  Sales Convention pictures would be good for the high-end of extraversion; Police taking down violent offenders for low levels of agreeableness.  The potential for pre-hire selection, especially using to add to Computer-Adaptive Testing item banks is tremendous.

c) Culture & Climate

Static pictures may be difficult to identify that reflect various organizational cultural differences, but videos could certainly be used to assess these. 

As optimistic as I am about the potential for picture-based items to take a larger role in organizational assessment, I recognize there are also downsides.  First, while digital cameras are cheap, actors may not be.  If you can find existing workplaces where you can take these pictures, it may help you avoid hiring actors for static pictures, but perhaps not for videos that could really suffer with amateur actors.

Second, one New Zeland user of the PHODA complains that if the photographs are context-specific, they can loose value in other contexts.  I remember once when I worked for AT&T Microelectronics, we hired Wally Borman to redo his 1970’s era rater training videos because while the content was good, the actors wore sideburns, bell-bottoms and leisure suits.  This was never going to be very persuasive as “cutting edge” to managers in a bleeding-edge semiconductor factory (computer chips).

Do you see the same potential for photograph-based items as The Scientific Leader?

, , , , , , , , , , , , , , , , , , , , , ,


Madoff Destroys $50 Billion with “Giant Ponzie Scheme”

Bernie Madoff is the latest in the series of senior executives to destroy value, this time with an apparent $50 billion dollar fraud, according to the Financial Times.  Madoff, a former Chairman of the NASDAQ stock market, on thursday admitted to his employees including his two sons that his operations were “all just one big lie” and “basically, a giant Ponzi scheme”.  The alleged fraud is the largest ever investor fraud ever blamed on a single individual.

Previously, I had written about the “Three Stooges of Operational Risk“, where I detailed senior executive destruction from Key Lay of Enron, Bernie Evers of Worldcom and most recently, Dick Fuld‘s follies with Lehman Brothers.  In two of those three I noted the dishonesty and fraud that accounted for their downfall similar to Madoff.  But unlike Madoff, they were less candid about thair fraud.  After Madoff’s brazen alleged admission, is there any uncertainty that leadership due dilligence is a critical part of the selection process of hiring senior executives?  Could it be any more clear that the pre-hire assessment procedure is a non-trivial subset of Enterprise Risk Management?

In fairness, these Industrial Organizational Psychology methods have their limitations.  No forecast could ever be perfect, or and even the best assessment procedures only account for 30-60% of the variance in job performance.  But it’s relatively rare that factors such as conscientiousness are used to screen executives – and conscientiousness highly predicts dishonest, and imprudent behavior in the workplace like that of Madoff.  With new methods from Rasch Measurement, Computer-Adaptive Testing, and an innovation from the Scientific Leader, “Inverted Computer Adaptive Testing” using Virtual Realtity, it’s increasingly difficult for people to fake or misrepresent themselves on these assessments. 

How much risk are you accepting when you use standard interviews to hire your employees?

, , , , , , , , , , , , ,

India’s Harvard to Use Computer Adaptive Testing

The premier management schools in India, the Indian Institutes of Management (IIMs) are migrating their common admission test to a computer adaptive test, using the same technologies deployed in the US to certify physicians, and offered by The Scientific Leader.  The Business Standard reports that IIM is seeking better information on the performance students, hence their desire to migrate to the most sophisticated form of human assessment available in the field of psychometrics.  IIM will use their CAT system to assess over 250,000 students for only 1,800 seats at IIM and other business schools.  With those large numbers taking the test, IIM will also benefit from adaptive testing’s improved question security – as every person gets a personalized test, and it’s rare that two people get the same questions with adaptive measurement.  CATs are unrelated to any sort of measurement of felines.

Are you tired of tedious tests?  Worried about the security of your high-stakes tests?  I encourage you to check out The Scientific Leader’s free whitepaper on the applications of Computer Adaptive Testing to leadership assessment.

, , , , , , , , , , ,

Successful Wal-Mart CEO Succession Plan

The world’s largest retailer, Wal-Mart, announced that it has chosen Mike Duke to succeed H. Lee Scott Jr. as CEO and President, according to the New York Times.  I’m surprised that the pundits wouldn’t anticipate this succession move.  Scott, 59, is providing a responsible, smooth transition at a time when Wal-Mart is positioned to be the top performer in the Dow Jones Industrial Average, and over the last eight years of his leadership was able to more than double its’ annual revenue to $378.8 billion, according to Bloomberg.  And he’s not abandoning ship – he’ll remain Chairman of the board’s executive committee.  This appears to be a textbook excellent succession:

  1. Wal-mart grew it’s leadership bench strength sufficiently to provide an internal candidate
  2. The transition is at a time when the firm is doing well.  Further, as the US goes deeper into a recession, consumers are likely to switch from higher priced goods to shopping at Wal-Mart.
  3. Mr. Duke has held a variety of roles, including most recently running their challenging international operations.  These experiences give him unique, general management perspective on making decisions to help the complex multinational win in the brutally competitive low-priced retail goods segment.
  4. Mr. Scott will continue to be involved with the company, and provide coaching and counsel to ensure a smooth transition.

Even though some analysts are surprised, the NYSE took the news well, and Wal-Mart gained 2.8% on the news.  Those of you who are a bit surprised with this might appreciate an excellent book The Scientific Leader recommends by Robert Barner, “Bench Strength: developing the depth and versatility of your organization’s leadership talent“. It’s a good blend of science and practice, and is very accessable to lay leaders.  Do you have a succession plan with sufficient bench strength to provide this sort of smooth transition? How evidence-based is your information on this bench strength?  Is it more than a company-sanctioned documented rumor?

, , , , , , , , ,

Market Yodels for New Yahoo CEO

Yahoo’s stock rose 17% today, after CEO and co-founder Jerry Yang announced his decision to step down from the top post., according to Bloomberg.

Yang was a leader who co-founded Yahoo, which in its’ hayday was one of the premier new economy internet companies.  But in the last decade, Yahoo has not maintained it’s innovative streak, and lost search engine share to Google and Microsoft.   Yang’s brief 17-month tenure was marked by failed mergers and partnerships that caused the stock price to plunge, reports CNN.  The New York Times reports that one of his highest profile failures was rejecting a $44B acquisition offer from Microsoft last spring.  Could leadership science have predicted this with Yang at the lead?

Perhaps.  The “Full Range” leadership model by Bass & Avolio is supported by 20+ years of research showing that charismatic, transformational leaders outperform their counterparts.  Their work strongly shows that followers walk over hot coals for leaders who connect with their people as individuals, provide intellectual stimulation and walk their talk.  According to CNET, Yang’s early November performance at a Web 2.0 summit was described as a “train wreck”, “self-delusional” and “making a mockery of the vaunted company he helped create”.  They argue that Yang can’t sell his vision inside or outside the company, that

 “He lacks the out-sized personality and charisma that is needed to inspire confidence in battles for the soul of a company.  He has to convince employees, shareholders, customers, and partners
that no matter how difficult the situation, he can lead Yahoo to the
promised land. Think reality-distortion field Steve Jobs, no-software
Marc Benioff, dancing bear Steve Ballmer, the disarming Howard
Stringer, the professorial Eric Schmidt, or the preacher John Chambers.
Bill Gates doesn’t have the most charismatic or endearing personality,
but he manages to control interviews, delivering the messages he wants.

Because there is substantial evidence that the sorts of charismatic leadership factors predict leadership performance, or in Yahoo’s case, non-performance, The Scientific Leader’s computer-adaptive assessments of leadership are based on Bass & Avolio’s work.  I invite you to make your own judgment of Mr. Yang by reviewing this link of one of his last interviews before stepping down.  Is this interview a trainwreck?  Is he charismatic to you?

Technorati Tags: , , , , , , , , ,

Improved Leadership Due Diligence White Paper

Enron's Ken Lay

Key Lay of Enron

I’m very grateful to all the helpful words of encouragement about my free whitepaper, “The Three Stooges of Operational Risk: Advances in Leadership Due Diligence and Rasch Measurement“.  I made numerous additions to the white paper including:

a) Detailed treatment of the evidence-based definition of leadership

b) Virtual Reality-based Assessment

c) Six Sigma / Industrial Engineering-inspired improvements to detecting faking

and much more.  I would greatly appreciate more feedback.  Is the business case compelling to you?  How much do traditional methods hurt your ability to assess and mitigate executive risks?

Technorati Tags: , , , , , , , , , ,

Full Employment Act for Financial Risk Managers

CFO.com reports that the one small silver lining in the recent financial crisis is that financial professionals, especially the top job, are more important than ever.  Some have called it a “full employment act” for financial risk managers, who seek to mitigate the downside, and exploit the upside, to “get paid” for taking appropriate risks.

The Institute for Financial Analysts provide a certification program for those seeking Financial Risk Management Skills (FRM), and for $2,500 include instruction in the following areas:

Basel II that involve current banking laws and regulations to create international standards.
Quantitative Analysis, including measurement and modeling of risk
Enterprise Risk Management to manage risk (Avoid, Reduce, Share/Insure, Accept).

Recommended books include The Professional Risk Manager’s Handbook and the Financial Risk Manager Handbook

These address the technical knowledge requirements to be an effective Risk Manager, however, according to the Open Source O*Net online job analysis information include other critical job tasks and skills including personality traits such as integrity, and stress tolerance (Low Neuroticism).  I wonder if pre-employment risk management assessments are used to manage Risk Manager risks?

Technorati Tags: , , , , , ,