Hiring

Improving Multiple-Choice Assessments by Limiting Time

Standardized, multiple-choice assessments frequently come under fire because they test rote skills, rather than practical, real-world application.  Although this is a gross over-generalization failing to account for the cognitive-complexity the items (questions) are written to, standardized assessments are designed to evaluate what a person knows, not how well they can apply it.  If that were the end of the discussion, you could be forgiven in assuming standardized testing is poor at predicting real-world performance or differentiating between novices and more seasoned, experienced practitioners.  However, there is another component that, when added to standardized testing, can raise assessments to a higher level: time.  Time, or more precisely, control over the amount of time allowed to perform the exam, can be highly effective in differentiating between competence and non-competence.

The Science Bit

Research in the field of expertise and expert performance suggests experts not only have the capacity to know more, they also know in a way differently than non-experts; experts exhibit different mental models than novices (Feltovich, Prietula, & Ericsson, 2006).  Mental models represent how individuals organize and implement knowledge, instead of explicitly determining what that knowledge encompasses.  Novice practitioners start with mental models representing the most basic elements of the knowledge required within a domain, and their mental models gradually gain complexity and refinement as the novice gains practical experience applying those models in real world performance (Chase & Simon, 1973; Chi, Glaser, & Rees, 1982; Gogus, 2013; Insch, McIntyre, & Dawley, 2008; Schack, 2004).

While Chase and Simon (1973) first theorized that the way experts chunk and sequence information mediated their superior performance, Feltovich et al. (2006) suggested these changes facilitated experts processing more information faster and with less cognitive effort contributing to greater performance. Feltovich et al. (2006) noted this effect as one of the best-established characteristics of expertise and demonstrated in numerous knowledge domains including chess, bridge, electronics, physics problem solving, and medical applications.

For example, Chi et al. (1982) determined that the way novices and experts approach problem-solving in advanced physics was significantly different despite all subjects having the same actual knowledge necessary for the problem solution; novices focused on surface details while experts approached problems from a deeper, theoretical perspective.  Chi et al. also demonstrated the novice’s lack of experience and practical application contributed to errors in problem analysis requiring more time and effort to overcome. While the base knowledge of experts and novices may not differ significantly, experts appear to approach problem solving from a differentiated perspective allowing them more success in applying correct solutions the first time and recovering faster when initial solutions fail.

In that vein of thought, Gogus (2013) demonstrated that expert models were highly interconnected and complex in nature, representing how experience allowed experts the application of greater amounts of knowledge in problem solving.  The ability for applying existing knowledge with greater efficiency augments the difference in problem-solving strategy demonstrated by Chi et al. (1982).  Whereas novices apply problem-solving approaches linearly one at a time, experts evaluate multiple approaches simultaneously in determining the most appropriate course of action.

Achieving expertise is, therefore, not simply a matter of accumulating knowledge and skills, but a complex transformation of the way experts implement that knowledge and skill (Feltovich et al., 2006). This distinction provides clues into better implementing assessments to differentiate between expert and novice: the time it takes to complete an assessment.

Cool Real-World Example Using Football (Sorry. Soccer)

In an interesting twist on typical mental model assessment studies, Lex, Essig, Knoblauch, and Schack (2015) asked novice and experienced soccer players to quickly and accurately decide the best choice of tactics (either “a” or “b”) given a video image of a simulated game situation.  Lex et al. used eye-tracking systems to measure how the participants reviewed the image, as well as measuring their accuracy and response time.  As one would expect, the more experienced players were both more accurate in their responses, as well as quicker. Somewhat surprising was the reason experienced players performed faster.

While Lex et al. (2015) determined both sets of players fixated on individual pixels in the image for nearly the same amount of time, experienced players had less fixations and observed less pixels overall.   Less experienced players needed to review more of the image before deciding, and were still more likely to make incorrect decisions.  On the other hand, more experienced players, although not perfect, made more accurate decisions based on less information.  The difference in performance was not attributable to differences in basic understanding of tactics or playing soccer, but the ability of experienced players to make better decisions with less information and taking less time.

The Takeaway

Multiple-choice, standardized assessments are principally designed to differentiate what people know, with limited ability to differentiate how well they can apply that knowledge in the real world.  Yet, it is also well-established that competent performers have numerous advantages leading to better performance in less time.    If time constraints are actively and responsibly constructed as an integral component of these assessments, they may well achieve better predictive performance; they could do a much better job of evaluating not just what someone knows, but how well they can apply it.

 

References

Chase, W. G., & Simon, H. A. (1973). The mind’s eye in chess. In Visual Information Processing (pp. 215–281). New York, NY: Academic Press, Inc. http://doi.org/10.1016/B978-0-12-170150-5.50011-1

Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1, pp. 7–75). Hillsdale: Lawrence Erlbaum Associates.

Feltovich, P. J., Prietula, M. J., & Ericsson, K. A. (2006). Studies of expertise from psychological perspectives. In The Cambridge handbook of expertise and expert …. New York, NY: Cambridge University Press.

Gogus, A. (2013). Evaluating mental models in mathematics: A comparison of methods. Educational Technology Research and Development, 61(2), 171–195. http://doi.org/10.1007/s11423-012-9281-2

Insch, G. S., McIntyre, N., & Dawley, D. (2008). Tacit Knowledge: A Refinement and Empirical Test of the Academic Tacit Knowledge Scale. The Journal of Psychology, 142(6), 561–579. http://doi.org/10.3200/jrlp.142.6.561-580

Lex, H., Essig, K., Knoblauch, A., & Schack, T. (2015). Cognitive Representations and Cognitive Processing of Team-Specific Tactics in Soccer. PLoS ONE, 10(2), 1–19. http://doi.org/10.1371/journal.pone.0118219

Schack, T. (2004). Knowledge and performance in action. Journal of Knowledge Management, 8(4), 38–53. http://doi.org/10.1108/13673270410548478

The Versatilist Vs. the Peter Principle

It is surprising how few are familiar with the Peter Principle.  This is most disturbing in the areas of business and organizational psychology as it speaks directly to the source of innumerable challenges for organizational success.  It should be a risk factor in talent management, succession planning, and organizational compensation systems.  Most of all, organizations should look at ways to circumvent this process; most notably, organizations should consider how Verstalists can thwart the Peter Principle.

The Peter Principle

“In a Hierarchy Every Employee Tends to Rise to His [Her] Level of Incompetence” (Peter & Hull, 1969, p. 25).

In action, the Peter Principle states a simple inevitability.  If you are good at your job, you get promoted.  If you are good at the new job, you get promoted again.  This continues until you take on a job for which you are not well suited; i.e. incompetent.  Having reached your level of incompetence, you no longer get promoted, but stay within the job you are least capable of performing well.   Taken to its ultimate conclusion, organizations eventually become dominated by leaders with the least capability to do their job.

Although published originally in 1969 as a tongue-in-cheek exposition on incompetence within human organizations, and supported by fictitious research, the Peter Principle continues to be debated amongst practitioners and academics.  It has been lambasted as unscientific (something it never purported to be) and crass overgeneralization, as well as an insightful source of legitimate inquiry.  The staying power of the Peter Principle maybe that its simplicity and succinctness, aligns with human experience and explains why so many organizations manage to do such stupid things.

The Value of the Peter Principle Perspective

Despite the limited academic basis for the Peter Principle, it manages to highlight a particular problem – why do we use promotion as a means of reward for competent performance (Fairburn & Malcomson, 2001)?  Doing so fails to consider two very fundamental truths: competence is domain specific, and management is a very specific skill. These fundamental truths conflict with the way most organizations reward and promote people.  Failure to acknowledge these promotes inefficiency, turmoil, and perpetuates the validity of the Peter Principle.

First, few organizations design their job families to reward and promote people for simply getting better and more efficient at the job they do.  Moving from job-specialist level 1, to job-specialist level 2, often requires doing different things, instead of doing the same things better.  We reward people, not for being good at their job, but for taking on new roles they have never done before, not proven they are capable of, and promoting constant change rather than long-term competence.   As soon as people demonstrate competence, we move them.

Second, organizations fail to realize management and leadership skills as a unique job all to themselves.  Being a good engineer says nothing about your ability to be a good engineering manager; however, good leadership skills can be a boon regardless of the function or industry. While understanding the jobs people need to perform is beneficial, leaders do not have to be competent in all the job functions they lead.  Promoting people who are competent in their job but shown no competence for leadership to positions of leadership, once again, promotes inefficiency and disruption.  Not only do you lose a competent performer in their prior role, but you may very well promote incompetent leadership.

Versatilists to the Rescue

Versatilists rarely run afoul of the Peter Principle.  First, versatilists are rarely promoted very high within most organizations, because they do not stay within any specific domain very long (something HR departments seem to think predicts success).  Second, because versatilists are deeply knowledgeable about many domains, they are keenly aware of what they are, and more importantly are not, capable of doing.  As such, versatilists without the desire or capability to lead will not pursue those opportunities.  Versatilists could be the savior for organizations looking to thwart the Peter Principle, but it will require HR to change their perspective on talent acquisition and development.

In terms of talent acquisition, HR and recruiting need to look beyond the experience requirements they believe are required for a job, and begin looking at the actual skills.  Far too often, organizations are looking for years of single domain experience (like engineering and software development) for roles that don’t necessarily require that experience (like leading engineering and software development teams).   The skills themselves are more important than the domain in which they were developed.   This is important for strategic innovation in particular, where having new perspectives brought to the job can be highly valuable.  A versatilists with leadership capability can quickly adapt to new industries and environments, while also bringing a host of new skills.

HR/Recruiting should also consider the quantity and quality of performance, rather than simply the length of performance, when looking at promotions or new hires.  Comparing two candidates for a position, a candidate who has shown success in multiple assignments and multiple environments over numerous years, should be preferred to one that has shown success in a single domain over the same time.  The candidate with multiple, differentiated success is much more likely to be successful in the new job as well; the one in a single domain is ripe to be reaching their level of incompetence.  Success in adapting to new environments is a skill companies should value, but don’t.

In terms of talent development, HR needs to create ways of rewarding specialists who do their job increasingly well over years of dedication without using promotions, while appreciating the versatilists who thrive in taking on new roles.  Promotions should not be the only means of rewarding top performers; bonuses and incentives should be used to drive continued competence building.  Promotions should only be used to expand and diversify the experiences of those already proving their ability to adapt and succeed in new roles.   HR needs to look beyond narrow definitions to find the people most likely to succeed, not those that have just been doing it longer.

As companies continue to struggle with market volatility, disruptive innovation, and dramatic shifts in business models, versatilism should be the new standard of performance.  What good is someone who has ten years’ experience in business models and practices that no longer hold true? Perhaps a new principle, the Versatilist Veracity should succeed the Peter Principle:

“Without Versatilists, a Hierarchy Tends to Become Incompetent”

 

References

Fairburn, J. a, & Malcomson, J. M. (2001). Performance, promotion, and the Peter Principle. Review of Economic Studies, 68(1), 45–66. http://doi.org/10.1111/1467-937X.00159

Peter, L. J., & Hull, R. (1969). The Peter Principle. Cutchogue, N.Y.: Willima Morrow & Co., Inc.

Diversity is not just a social issue, it is an economic one.

The problem with treating diversity as only a social-justice issue is that social issues rarely get solved without demonstrating how they indirectly affect all people, not just the disenfranchised.  All you must do is look at the history of social corporate responsibility (CSR) to see this effect (O’Toole & Vogel, 2011).  CSR was treated with mostly lip-service until two things were demonstrably clear in the marketplace: 1) consumer trends were changing to favor organizations demonstrating CSR principles; and, 2) CSR (or sustainable) practices made economic sense by reducing waste, improving operations, and elevating brand.  While the messaging is about being socially conscious, CSR business models can lead to competitive differentiation, which leading to profits. The only real academic argument against CSR is the implied altruistic nature of most CSR proponents; every corporation engaging in CSR generates either direct or indirect economic profit from those actions, meaning CSR is nothing more than “enlightened self-interest” (Smith, 2003).  As much as we should care about social issues, until they affect us directly, critical mass is not achieved towards solving them.

Diversity is no different.  We, as a collective species, should promote diversity (religious, nationality, sex, age, values, etc.) simply because it is ethical and just. However, because of the numerous permutations of bias, no single group of disenfranchised gains sufficient support to make true change solely on the basis of justice.   Unless we can demonstrate how biases affect everyone, progress will remain slow, or non-existent.  The best way to combat isses of diversity is through developing the enlightened self-interest of the greater society.

Fortunately, there is tremendous support that diversity is the key to economic profit and productivity benefiting everyone, including those who are not victims of bias.  It is no accident that our national headlines, business articles, and social commentary are inundated with stories about both diversity as well as innovation.  These two concepts are intimately conjoined. Without diversity of experience, thought, and perspective, innovation does not happen; without innovation, society will no longer grow and prosper, but decline.  This affects everyone.

Innovation is the only true competitive differentiator in today’s world economy (Drucker, 1992; Friedman, 2006; Salchow Jr., 2016; Teece, 1998, 2004). Whether the innovation is a means to increase organizational efficiency, develop new business models, or an innovative product, the days of competing solely on accumulated land, capital, equipment, or market dominance are long over.  Those who don’t innovate, fail in the long-run. This affects people, companies, communities, and countries not just certain individuals. The inability to adapt to the global economy has decimated entire regions in the U.S. from miners, to steal producers, to manufacturers.   The number of companies and individuals directly affected is miniscule to the number of companies, individuals, and communities that have collapsed indirectly from these failures.  Lack of innovation capability affects us all.

Yet, we know that diversity in perspective, knowledge, experience, and capabilities is a foundation of innovation (Gladis, 2017). We know that diversity drives innovation (Niebuhr, 2010; Parrotta, Pozzoli, & Pytlikova, 2014), and creates economic rents, productivity, and success (Beck & Walmsley, 2012; Crook, Todd, Combs, Woehr, & Ketchen, 2011; Kim & Ployhart, 2014). Without diversity, we cannot hope to innovate because innovation is all about seeing things from a different perspective, a different value structure, a different life experience, a different cognitive lens.  It is through exploring and evaluating these differences that we see new possibilities, new solutions, and new ways of moving forward as companies, communities, and societies.  Diversity forces us to challenge what we think we know, and that leads to innovation.

It is sad that at a time when collaboration and access to diverse perspectives is so easy, we have instead taken to divisiveness, to segregation. We seek the illusionary safety of the known and miss the forest for the trees that don’t look, act, talk, or believe like us.  However, if we fail to see how diversity is an asset, not a liability, we fail our society. We fail, not because we violate the social contract, but because we will bankrupt society.  We fail by succumbing to what we believe is, rather than seeing what could be.  Without innovation, driven by diversity, we become static and eventually decline (Second Law of Thermodynamics anyone?).

Diversity is an economic imperative, not just a social one. The best way to secure your own future, is to seek out and embrace diversity. It is in our own self-interest.

References

Beck, J. W., & Walmsley, P. T. (2012). Selection ratio and employee retention as antecedents of competitive advantage. Industrial and Organizational Psychology, 5(1), 92–95. http://doi.org/10.1111/j.1754-9434.2011.01410.x

Crook, T. R., Todd, S. Y., Combs, J. G., Woehr, D. J., & Ketchen, D. J. J. (2011). Does human capital matter? A meta-analysis of the relationship between human capital and firm performance. Journal of Applied Psychology, 96(3), 443–456. http://doi.org/10.1037/a0022147

Drucker, P. F. (1992). The post-capitalist world. Public Interest, 109(Fall 1992), 89–101. Retrieved from http://www.nationalaffairs.com/

Friedman, T. L. (2006). The world is flat: A brief history of the twenty-first century. New York, NY: Farrar, Straus and Giroux.

Gladis, S. (2017). The Formula for Achieving Innovation. TD: Talent Development, (February).

Kim, Y., & Ployhart, R. E. (2014). The effects of staffing and training on firm productivity and profit growth before, during, and after the Great Recession. The Journal of Applied Psychology, 99(3), 361–89. http://doi.org/10.1037/a0035408

Niebuhr, A. (2010). Migration and innovation: Does cultural diversity matter for regional R&D activity? Papers in Regional Science, 89(3), 563–585. http://doi.org/10.1111/j.1435-5957.2009.00271.x

O’Toole, J., & Vogel, D. (2011). Two and a half cheers for conscious capitalism. California Management Review, 53(3), 60–76. http://doi.org/10.1525/cmr.2011.53.3.60

Parrotta, P., Pozzoli, D., & Pytlikova, M. (2014). The nexus between labor diversity and firm’s innovation. Journal of Population Economics, 27(2), 303–364. http://doi.org/10.1007/s00148-013-0491-7

Smith, H. J. (2003). The shareholders vs. stakeholders debate. MIT Sloan Management Review, 44(4), 85–90. Retrieved from http://sloanreview.mit.edu/

Teece, D. J. (1998). Capturing value from knowledge assets: The new economy, markets for know-how, and intangible assets. California Management Review, 40(3), 55–79. http://doi.org/10.2307/41165943

Teece, D. J. (2004). Knowledge and competence as strategic assets. Handbook on Knowledge Management 1: Knowledge Matters, 40(3), 129–152. http://doi.org/http://dx.doi.org/10.1007/978-3-540-24746-3_7

Using Mental Models to Identify Expertise

Research in the field of expertise and expert performance suggest experts not only have the capacity to know more, they also know it differently than non-experts; experts employ different mental models than novices (Feltovich, Prietula, & Ericsson, 2006). While it remains unclear how antecedents directly affect the generation of mental models, the relationship between mental models and performance is demonstrated across multiple domains of research (Chi, Glaser, & Rees, 1982; Feltovich et al., 2006). Unlike attempts to directly elicit the antecedents of performance that may, or may not, contribute to future performance, the mental models of experts show stable and reliable differences in expert performance without requiring the artificial constructs of tacit knowledge measurements (Frank, Land, & Schack, 2013; Land, Frank, & Schack, 2014; Lex, Essig, Knoblauch, & Schack, 2015; Schack, 2004, 2012; Schack & Mechsner, 2006). The potential to accurately, easily, and quantifiably define job-related expertise is an organizational opportunity for both the accumulation as well as the management of talent.

What are Mental Models

Based on information processing and cognitive science theories, mental models are the cognitive organization of knowledge in long-term memory (LTM) developed through learning and experience (Chase & Simon, 1973; Chi et al., 1982; Gogus, 2013; Insch, McIntyre, & Dawley, 2008; Schack, 2004).  Mental models represent how individuals organize and implement knowledge, instead of explicitly determining what that knowledge encompasses.  Novice practitioners start with mental models consisting of the most basic elements of knowledge required, and their mental models gradually gain complexity and refinement as the novice gains practical experience applying those models in the real world (Chase & Simon, 1973; Chi et al., 1982; Gogus, 2013; Insch et al., 2008; Schack, 2004).  Consequently, achieving expertise is not simply a matter of accumulating knowledge and skills, but a complex transformation of the way knowledge and skill is implemented (Feltovich et al., 2006).  This distinction, between what the individual knows and how the individual applies that knowledge has theoretical as well as practical importance for use in human assessment.

Mental models capture important aspects that plagued prior attempts to assess human capital performance.  In contrast to prior assessment methods, differences in mental models propose to demonstrate differences in the way individuals apply knowledge cognitively, rather than differences in the knowledge itself  (Chi et al., 1982; Gogus, 2013; Insch et al., 2008).   The significance of these findings is the implication of a measurable basis for the difference in performance between expert and novice, substantiating mental models as the quintessential construct defining the difference between the knowledge an individual has versus how the individual applies that knowledge.

Evaluating mental models from a practical perspective, mental models clearly differentiate between expert and non-experts. Chase and Simon (1973) first theorized that the way experts chunk and sequence information mediated their superior performance. Simon and Chase found grand master chess players’ superior performance resulted from recalling more complex information chunks.  These authors demonstrated that both experts and novices could recall the same number of chunks, but the chunks of novices contained single chess pieces whereas the chunks of experts contained meaningful chess positions composed of numerous pieces.  Simon and Chase further showed this superior performance to be context sensitive and domain specific as grand masters were no better than novices at recalling random, non-game specific piece constellations and showed no better performance in non-chess related memory.  The domain dependency indicates mental models of performance are not universal predictors but have job-related specificity making them ideal for assessment.

The observation that expert and novices store and access domain-specific knowledge differently spawned research theorizing quantitative, measurable differences in knowledge representation and organization might differentiate expert performance from non-expert performance (Ericsson, 2006). This research continues to substantiate increased experience and practice as the driver in the development of larger, more complex cognitive chunks (Feltovich et al., 2006). Feltovich et al. (2006) noted this effect as one of the best-established characteristics of expertise and demonstrated in numerous knowledge domains including chess, bridge, electronics, physics problem solving, and medical applications.  Feltovich et al. suggested these changes facilitated experts processing more information faster, with less cognitive effort thus contributing to greater performance.

Evolution of Mental Model Evaluation

The conceptualization of evaluating expert performance in academic and business domains already indicates the importance of mental model differences (Chi et al., 1982; Insch et al., 2008; Jafari, Akhavan, & Nourizadeh, 2013).  The general acceptance of mental models as a critical discriminator of performance has driven a deeper focus on the nature and structure of these differences instead of the specific knowledge they represent (Gogus, 2013).  This evolution of mental model evaluation, from a theoretical construct to a quantitative measure, mirrors the evolution away from what individuals know, towards how individuals utilize that knowledge.

Studies of expertise and expert performance demonstrate the dramatic differences in the way experts and novices organize knowledge in complex physics problem solving a (Chi et al., 1982). Chi et al. (1982) utilized cluster analysis to show differences in the way experts and novices structure their knowledge; however, mental models were only one of several ways in which the authors analyzed expert and novice differences.

Acknowledgment of these differences in mental representations rationalized the use of mental models in constructing more traditional tacit knowledge measures (Insch et al., 2008). Insch et al. (2008) approached tacit knowledge measures through evaluation of the actions individuals performed, acknowledging tacit knowledge was inherently how individuals use knowledge, not necessarily what knowledge they had.  In taking this approach, the authors focused on the mental schemas that directed behavior instead of the antecedent values, beliefs and skills that contribute to performance.  The focus on schemas as the driving factor in performance is notable as divergent from prior tacit knowledge measures; however, Insch et al. did not attempt measuring and comparing resultant mental models explicitly.

More recently, Jafari et al. (2013) looked to elicit and visualize the tacit knowledge of Iranian autoworkers concerning their knowledge of organizational strengths.  The uniqueness of this study was the use of quantifiable measures of individual tacit knowledge for comparison between groups of individuals and purported experts, as well as the use of graphs to visualize the results for each group.  Jafari et al. stipulated differences in mental models as an indication of differences between novice and expert workers but focused on the content rather than the structure of the mental model.  The authors further operationalized the quantitative measures as differences in what the individuals knew, and not how they utilized or implemented the knowledge. This approach advanced the use of mental models in the identification of expert knowledge, yet failed to identify how these models differ regarding application or structure.

Other researchers focused more on the differences in comparative mental models than the specific knowledge represented within the models (Gogus, 2013).  In evaluating the applicability and reliability of different methods of eliciting and comparing mental models, Gogus (2013) suggested the theoretical and methodological approach to the analysis of mental models is independent of the domain of knowledge.  Gogus replicated and contrasted the use of two different methodologies for externalization and measurement of mental model differences.  Of particular note, the author focused on contrasting the features of mental models instead of on the specific knowledge, experience, attitude, beliefs, or values of participants.  These efforts further support differences in mental models as being more dependent on the tacit rather than explicit knowledge of the individual. Since mental models are inherently domain specific and often contain the same base explicit knowledge, structural differences in the mental models between experts and novices are more indicative of the differences in performance.

Research in the area of sports psychology has similarly focused on developing reliable means of differentiating mental models of individuals to differentiate performance and diagnose performance problems.  Distinct differences in the mental models between experts and novices have been documented across multiple action-oriented skills including tennis (Schack & Mechsner, 2006), soccer (Lex et al., 2015), volleyball (Schack, 2012), and golf (Frank et al., 2013; Land et al., 2014). Schack and Mechsner (2006) demonstrated how differences in the mental models of the tennis serve related to the level of expertise.  Lex et al. (2015) evaluated the differences in the mental models of team-specific tactics between players of varying levels of experience.  Less experienced players averaging 3.2 years of experience (n = 20, SD = 4.2) generated mental models viewing team-tactics broadly as either offensive or defensive.  More experienced players averaging 17.3 years of experience (n = 18, SD = 3.3), further differentiated offensive and defensive tactics into smaller groups of related actions.  For instance, more experienced players further segmented defensive tactics into actions for pressing the offense, and returning to standard defense.

Focus on the specific differences in the structure of mental models has not only proven effective in differentiating expert and novice performance but also provided insight into effective training regimens (Frank et al., 2013; Land et al., 2014; Weigelt, Ahlmeyer, Lex, & Schack, 2011).  Frank et al. (2013) compared the models of novice performers to those of experts prior to and following a training intervention.  The authors experimentally evaluated two randomly assigned groups of participants with no former experience in performing a golf putt.  With the exception of an initial training video provided to all participants, none received any training or feedback.  The experimental group participated in self-directed practice over a three-day period, while the control group did not practice at all.  Frank et al. found the mental models of participants subjected to practice evolved, becoming more similar to expert mental models than participants in the control group.  Since the formal knowledge of all participants remained the same, the outcome of this study further suggests the structure of individual mental models is dependent on the experience and tacit knowledge of the individual.

The Opportunity

The use of mental models to identify expertise shows great promise. Variations in mental model construction differentiate clearly between expert and novice performers across numerous domains of knowledge.  Furthermore, methodologies highlighting the structural differences between the mental models of experts and novices show promise in the development and evaluation of training regimens.  As a result, the development of human capital assessments based on the measurement of the structural differences between mental models represents a strategic opportunity for organizations to improve the quality of human capital selection as well as the development and assessment of existing human capital.

References

Chase, W. G., & Simon, H. A. (1973). The mind’s eye in chess. In Visual Information Processing (pp. 215–281). New York, NY: Academic Press, Inc. http://doi.org/10.1016/B978-0-12-170150-5.50011-1

Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1, pp. 7–75). Hillsdale: Lawrence Erlbaum Associates.

Ericsson, K. A. (2006). An introduction to Cambridge handbook of expertise and expert performance: Its development, organization, and content. In The Cambridge handbook of expertise and expert …. New York, NY: Cambridge University Press.

Feltovich, P. J., Prietula, M. J., & Ericsson, K. A. (2006). Studies of expertise from psychological perspectives. In The Cambridge handbook of expertise and expert …. New York, NY: Cambridge University Press.

Frank, C., Land, W., & Schack, T. (2013). Mental representation and learning: The influence of practice on the development of mental representation structure in complex action. Psychology of Sport and Exercise, 14(3), 353–361. http://doi.org/10.1016/j.psychsport.2012.12.001

Gogus, A. (2013). Evaluating mental models in mathematics: A comparison of methods. Educational Technology Research and Development, 61(2), 171–195. http://doi.org/10.1007/s11423-012-9281-2

Insch, G. S., McIntyre, N., & Dawley, D. (2008). Tacit Knowledge: A Refinement and Empirical Test of the Academic Tacit Knowledge Scale. The Journal of Psychology, 142(6), 561–579. http://doi.org/10.3200/jrlp.142.6.561-580

Jafari, M., Akhavan, P., & Nourizadeh, M. (2013). Classification of human resources based on measurement of tacit knowledge. The Journal of Management Development, 32(4), 376–403. http://doi.org/http://dx.doi.org/10.1108/02621711311326374

Land, W. M., Frank, C., & Schack, T. (2014). The influence of attentional focus on the development of skill representation in a complex action. Psychology of Sport and Exercise, 15(1), 30–38. http://doi.org/10.1016/j.psychsport.2013.09.006

Lex, H., Essig, K., Knoblauch, A., & Schack, T. (2015). Cognitive Representations and Cognitive Processing of Team-Specific Tactics in Soccer. PLoS ONE, 10(2), 1–19. http://doi.org/10.1371/journal.pone.0118219

Schack, T. (2004). Knowledge and performance in action. Journal of Knowledge Management, 8(4), 38–53. http://doi.org/10.1108/13673270410548478

Schack, T. (2012). Measuring mental representations. In G. Tenenbaum, R. Eklund, & A. Kamata (Eds.), Measurement in Sport and Exercise Psychology (pp. 203–214). Champaign, IL: Human Kinetics. Retrieved from http://www.uni-bielefeld.de/sport/arbeitsbereiche/ab_ii/publications/pub_pdf_archive/Schack (2012) Mental representation Handb

Schack, T., Essig, K., Frank, C., & Koester, D. (2014). Mental representation and motor imagery training. Frontiers in Human Neuroscience, 8(May), 328. http://doi.org/10.3389/fnhum.2014.00328

Schack, T., & Mechsner, F. (2006). Representation of motor skills in human long-term memory. Neuroscience Letters, 391(3), 77–81. http://doi.org/10.1016/j.neulet.2005.10.009

Weigelt, M., Ahlmeyer, T., Lex, H., & Schack, T. (2011). The cognitive representation of a throwing technique in judo experts – Technological ways for individual skill diagnostics in high-performance sports. Psychology of Sport and Exercise, 12(3), 231–235. http://doi.org/http://dx.doi.org/10.1016/j.psychsport.2010.11.001

Why you want to, but won’t, hire a Versatilist

The quality of an organization’s human capital is more important today than at any time before.  Global, dynamic markets eradicate the competitive advantages of capital, equipment, and land (Drucker, 1992; Friedman, 2006; Hayton, 2005; Teece, 2011).  Today, differentiation comes from combining undifferentiated inputs and resources in unique ways (Dutta, 2012; Reeves & Deimler, 2011; Teece, 2007, 2011, 2012; Teece, Pisano, & Shuen, 1997). As such, the source of competitive differentiation and strategic value is not having superior resources, but the skill and knowledge necessary to innovate.  One way to describe this organizational ability is dynamic capabilities (Teece, 2012). Dynamic capabilities characterize the organizational ability to sense and seize new opportunities and transform the organization, maintaining a competitive position.  Organizations with strong dynamic capabilities change and adapt to dynamic markets, are strong innovators, and build lasting strategic differentiation.  The only place this knowledge and skill resides is within the individuals working for the organization: human capital (Blair, 2002; Ployhart, Nyberg, Reilly, & Maltarich, 2014).

If we take the notion of dynamic capabilities and apply it to a person, instead of an organization, you get versatilists.  Versatilists are wired to sense and seize new opportunities, leverage new skills and abilities, and innovate who and what they are.  They are always changing and adapting to the world around them to become experts in new areas.  They don’t have access to different knowledge or methods of learning than other people, but they combine them in new ways to create new versions of themselves.  If organizations need dynamic capabilities to innovate and be successful, who better than versatilists to drive that effort.  This is why organizations should identify and recruit versatilists as employees.

Unfortunately, current recruiting and hiring strategies are ill aligned to this goal. Just look at your average Sr. level job description: 5 -7 years doing one thing with 10+ years in the same industry, with the same focus; another: 10 years in this job role, plus 5 years in specific industry. The job descriptions go on to list several dozen areas of knowledge and experience necessary to be considered a good fit.  These descriptions will use terms like “successful track record of”, “expertise in”, and “demonstrated experience with”. While this likely doesn’t sound out of place to many, especially those in HR and recruiting, it puts the job in a nice, little box tied with a bow.  The versatilists will rarely look twice for a couple of reasons.

First off, after 5-7 years doing the same thing, most versatilists are ready for the next challenge, not the next opportunity to do the same thing. The industry experience is less of an issue (although it’s still a bad way to get new ideas into your organization).  Versatilists don’t just adapt and change because of external forces; we’re not forced to go down a different path. We choose to do new things in new ways. There is an internal drive to know more, to do more, and to do it better.  Once a versatilists has become an expert in a role, we see little opportunity for growth, either personal or professional, and are naturally attracted to the next opportunity.

Second, unlike a generalist who tends to oversell their experience, versatilists, having become experts, generally undersell.  This is the Dunning-Kruger Effect in action (Dunning, Johnson, Ehrlinger, & Kruger, 2003; Kruger & Dunning, 1999).  According to this research, people tend to estimate their knowledge on any topic as at, or slightly above average.  Those with the least amount of actual knowledge overestimate grossly what they know (and don’t know they are doing it).  However, this works with experts as well, who underestimate their knowledge by assuming it is also just at, or slightly above average (this is sometimes referred to as imposter syndrome).  Because versatilists become experts in each of their chosen areas, even if you ask for “expertise” in that specific area, they will not feel qualified generally. This is further compounded when the job description suggests the candidate should be competent in dozens of areas.

Consequently, organizations limit their ability to hire versatilists the minute they draft a job description, making themselves unattractive to the very human capital they should really want.  Organizations cannot become innovative or develop dynamic capabilities, and yet hire based on check boxes and job descriptions of what the job has always been.  Instead, organizations should be hiring the people that can adapt and change the job to what it needs to be tomorrow.  Unless you change the way you recruit and hire, you’re more likely to hire someone without the skills you thought you needed and no capacity to develop the skills you really need.

 

References

Blair, D. C. (2002). Knowledge Management: Hype, Hope, or Help? Journal of the American Society for Information Science & Technology, 53(12), 1019–1028.

Drucker, P. F. (1992). The post-capitalist world. Public Interest, 109(Fall 1992), 89–101. Retrieved from http://www.nationalaffairs.com/

Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83–87. http://doi.org/10.1111/1467-8721.01235

Dutta, S. K. (2012). Dynamic capabilities: Fostering ambidexterity. SCMS Journal of Indian Management, 9(2), 81–91. Retrieved from http://search.proquest.com/

Friedman, T. L. (2006). The world is flat: A brief history of the twenty-first century. New York, NY: Farrar, Straus and Giroux.

Hayton, J. C. (2005). Competing in the new economy: the effect of intellectual capital on corporate entrepreneurship in high-technology new ventures. R&D Management, 35(2), 137–155. http://doi.org/10.1111/j.1467-9310.2005.00379.x

Kruger, J., & Dunning, D. (1999). Unskilled and Unaware of It : How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated. Journal of Personnality and Social Psychology, 77(6), 1121–1134. http://doi.org/10.1037/0022-3514.77.6.1121

Ployhart, R. E., Nyberg, A. J., Reilly, G., & Maltarich, M. a. (2014). Human capital Is dead; Long live human capital resources! Journal of Management, 40(2), 371–398. http://doi.org/10.1177/0149206313512152

Reeves, M., & Deimler, M. (2011). Adaptability: The new competitive advantage. Harvard Business Review, 89(7/8), 134–141. Retrieved from http://hbr.org/

Teece, D. J. (2007). Explicating dynamic capabilities: the nature and microfoundations of (sustainable) enterprise performance. Strategic Management Journal, 28(13), 1319–1350. http://doi.org/10.1002/smj.640

Teece, D. J. (2011). Dynamic capabilities: A guide for managers. Ivey Business Journal Online, 1. Retrieved from http://search.proquest.com/

Teece, D. J. (2012). Dynamic Capabilities: Routines versus entrepreneurial action. Journal of Management Studies, 49(8), 1395–1401. Retrieved from 10.1111/j.1467-6486.2012.01080.x

Teece, D. J., Pisano, G., & Shuen, A. (1997). Dynamic capabilities and strategic management. Strategic Management Journal, 18(7), 509–533. http://doi.org/10.1016/b978-0-7506-7088-3.50009-7