Rant

Pure opinion and perspective, not properly cited or fully researched.

Google Does Not Obviate “Knowing”

There is a strange notion making the rounds of social media in various forms, used to argue against traditional learning and assessment standards.  This reoccurring theme suggests the ubiquitous ability to leverage Google search, Wikipedia, or other online resources to find answers obviates the need to learn anything for yourself.  I.e., if we need to know something, we can just look it up in real-time and don’t need to waste time learning this information before we need it.  This theme has come up in discussions of our educational system curriculum, the supposed uselessness of standardized testing, and even in employee assessment criteria.

The Internet was never intended to be a replacement for independent knowledge.

Perhaps this is a special case of the Dunning-Kruger effect (Dunning, Johnson, Ehrlinger, & Kruger, 2003; Kruger & Dunning, 1999), but there are at least two clear reasons why access to knowledge is not equivalent to actually knowing it.  The first is a complete disconnect from the way human beings develop skill and competency.  The second is the assumption real-time knowledge, although ubiquitous, is accurate and will always be available.

Having Facts is Not “Knowing”

The most incongruous part of this idea is the assumption that knowledge is the result of just having a bunch of facts.  Thus, if you can just look up the facts, you have knowledge.  Unfortunately, unlike in the Matrix, human beings cannot simply download competence and expertise.

Learning something, and becoming good at it, is a process of building mental models on top of the foundation of rote facts

The study of experts and expert knowledge has well established the difference between experts and novices is not in what they know (the facts), but in how they apply those facts. It is based on how each fact fits with other facts or other pieces of knowledge. Expertise is the result of a process of integrating facts, context, and experience together and defining more refined and efficient mental models (Ericsson, 2006).  Learning something, and becoming good at it, is a process of building mental models on top of the foundation of rote facts.  This cannot be done without internalizing those facts.

In addition, returning to Dunning-Kruger, without building competence, individuals are incapable of discerning the veracity of individual facts.  Our ability to understand whether information is accurate, or of any substance, results from being able to rectify new information with our existing mental models and knowledge.  Those with less competence are the most unable to evaluate this information making them the most susceptible to not only accepting incorrect information as fact, but also of developing mental models incorrectly reflecting reality.

Limits of Ubiquitous Knowledge Access

Although those of use living in developed economies take ubiquitous access to knowledge for granted, this is not the case for all human beings, nor is it guaranteed to always exist.  It is estimated only about 50% of the world’s population is connected to the Internet, over two-thirds of which are in developed economies.  Even these figures bear further investigation, as those in developing countries with Internet access are far more likely to be connected by slower, less reliable means keeping their access from being truly ubiquitous.  Furthermore, while China contributes significantly to the world’s total Internet users, the Chinese government does not allow full, unrestricted access to the knowledge available via the Internet.  This leaves the number of people with true, ubiquitous access well below 50% of the population.

Even for those of us fortunate enough to have nearly ubiquitous access to an unrestricted Internet of knowledge, access is fragile.  Power outages as a result of simple failure, natural events, or even direct malice, can immediately render information inaccessible.  Emergency situations where survival might rely on knowledge also often exist outside the bounds of this seemingly ubiquitous access. Without a charge, or cellular connection, many find themselves ill-equipped to manage.

Dumbing Down our Society

The idea that access to knowledge is the same as having knowledge portends a loss of intellectual capital.  Whereas societies in the past have maintained control by limiting access to information, we are creating a future where control is maintained by delegitimizing and devaluing the accumulation of knowledge through full access to information.  We are positioning society to fail in the future because they will have not only become dependent on being spoon-fed information instead of actual learning, but will have also lost the ability to differentiate fact from fiction.

Not only is the idea that access to knowledge equates to having knowledge founded on shaky foundations lacking any kind of empirical basis, it undermines the actual development of knowledge

Although it would be nice to assume this is a dystopian view of the future, we are already seeing the effects of this process.  As social media becomes increasingly the way our society views the world around us, we can already see how ubiquitous access to information is affecting our perceptions of the world around us.  Without the ability to think critically, something only developed through the accumulation of knowledge and experience, in evaluating the real-time information we receive, our society is being manipulated into perspectives not of our own choosing, but the choosing of others.  We are losing the ability to process the information we receive and find ourselves increasingly caught in echo-chambers only presenting information supporting potentially incorrect world-views.

The Internet was never intended to be a replacement for independent knowledge.  It was developed to expand our ability to access information in the pursuit of developing knowledge and capability.  Not only is the idea that access to knowledge equates to having knowledge founded on shaky foundations lacking any kind of empirical basis, it undermines the actual development of knowledge.

 

 

Resources

Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83–87. http://doi.org/10.1111/1467-8721.01235

Ericsson, K. A. (2006). An introduction to Cambridge handbook of expertise and expert performance: Its development, organization, and content. In The Cambridge handbook of expertise and expert …. New York, NY: Cambridge University Press.

Kruger, J., & Dunning, D. (1999). Unskilled and Unaware of It : How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated. Journal of Personnality and Social Psychology, 77(6), 1121–1134. http://doi.org/10.1037/0022-3514.77.6.1121

 

Ecosystems Thinking for Social Change

Ecosystem, or “system” thinking is not necessarily about ecology, but uses an ecological metaphor to explore the interconnectedness of various aspects of any system (Mars, Bronstein, & Lusch, 2014).  This is a critical skill for business organizations to aid strategy and innovation.  It is also an area where versatilists often shine, because versatilists are uniquely adept at taking deep knowledge from one system and applying it to their understanding of new systems, often leading to unique insights.  However, that is not what this blog is about.  This is about how a lack of systems thinking is trapping our society into repeating the same issues repeatedly.  This is about how electing people who comprehend systems thinking might be a better means of bringing about social change.

The Heart of Systems Thinking

At the heart of systems thinking is to keep in mind that no problem, no solution, no individual exists in a vacuum.  The whole world is a set of interrelated systems that influence and affect those around it; changes in one system ripple throughout our entire society.  Systems thinking involves attempting to understand and evaluate any problem or solution within the context of the bigger picture.

For instance, take constraint theory (Tulasi & Rao, 2012).  Constraint theory suggests that any system or process is constrained by the least capable or least efficient step in the system; this is often equated with the “weakest link” idea that a chain is only as strong as the weakest link in the chain.  The idea behind constraint theory, however, suggests that if you fortify the weakest link (solving that problem), you have simultaneously created a new “weakest link” (formerly, the next-to-weakest link).  In addition, the newer, stronger link may also have other unintended consequences (maybe by making it stronger, you have also made it bigger, which affects some other function).  In essence, the process to creating a stronger chain is a never-ending task as each solution has ramifications.

In systems thinking, you must evaluate how a solution to one problem may create a new problem or change the dynamics of another system.  This potential new problem must also be evaluated to determine if it is a bigger problem than the one you are attempting to solve, or makes the solution you have proposed untenable.  Problem solving, like creating a stronger chain, is a never-ending process.  However, the intended result is improving the overall whole, ensuring one solution doesn’t create a bigger problem somewhere else.

Unfortunately, without systems thinking, we are failing to create an overall better society, but are remaining mostly stationary.  Solutions examined and evaluated within a vacuum, create ripples that, instead of moving us forward, keep us in a constant state.

Examples of Non-Systems Thinking Challenges

The worst part of failing to apply systems thinking to the problems of our society, is when the same groups of people argue for two independent solutions, which are counterproductive; i.e., when the same group argues for one solution that aggravates another problem they are trying to solve and vice-versa.  It is important to understand that, in and of themselves, the proposed solutions may be perfectly good solutions; it is only when you combine the system effects that issue become apparent.  It is also important to note that this is not an analysis of the merits of any particular solution or point of view.  There is no intent to endorse or oppose any of the individual solutions, simply to illustrate the systems effects of those solutions.

Immigration Reform and Free Trade Agreements

By itself, building a border wall, while economically questionable, is a perfectly legitimate solution to preventing illegal immigration via our southern US borer.  It is not the only solution of course, but it is a possible solution.  We can debate this one way or the other, but even opponents must admit that it is a solution whether they agree it is the right one or not.

Similarly, eliminating or significantly reducing free trade, particularly with low-cost labor countries like Mexico is a legitimate solution to reducing job-loss in the US via off-shore outsourcing by US companies and keeping US investment in the US.  Again, not the only solution, but certainly one way to address the issue.  We can once again debate this, but it must be accepted that it is a solution.

However, when put together, these solutions are counter-productive.  By eliminating the ability for Mexico to continue to develop and build their economic capability (by granting them easy access to US markets and US investment), their standard of living will likely decline.  A decline in standard of living (loss of jobs and the income from it) only perpetuates the growth of illicit enterprises (e.g. drugs) as well as makes illegally emigrating to the US more attractive.  As such, from a systems perspective, reversing free trade agreements will likely compound the issue of illegal immigration, as well as drug smuggling and other issues.  This places even bigger demands on the needs of border protection and immigration control.  These are misaligned solutions from a systems perspective.

Welfare Reform and Birth Control

Another counterproductive combination of arguments is simultaneously arguing for reducing the US welfare system, while simultaneously arguing to eliminate birth control options, including sex education and access to safe abortions.  Again, in and of their own, each of these arguments are perfectly reasonable and can be understood.   Without applying personal judgements on them, they are reasonable goals and can be respected.  From a systems perspective however, these are not isolated issues or goals, they have complex interactions which makes arguing for both less reasonable.   The only logical result of limiting sex education and access to birth control measures is an increase in women and children within the welfare system.  It is illogical to argue for both actions, even if either one of them in isolation can at least be recognized as reasonable.

Cyber Security and Encryption Strength

Just this last week, there were two articles published.  The first one detailed how Russian hackers have been targeting the personal (non-government) cell phones of NATO soldiers to track, intimidate, and spy on them.  The second one detailed the Department of Justice (DoJ) pushing for US technology firms to make it easier for law enforcement to access the encrypted (personal) devices of accused criminals.  Again, arguing for improved safety and security of our personal information, particularly through strong encryption is a reasonable solution to rampant cyber crime.  It is also reasonable to argue that law enforcement should be able to access the information they need to convict criminals for breaking the law.  Unfortunately, you cannot reasonably argue for both as the one comes at the cost of the other.  Arguing that we need to better protect our personal information from thieves, while simultaneously arguing to hobble encryption for government access are mutually exclusive goals.

Understanding the Bigger Picture is Essential

Systems thinking requires us to look at the proposed solutions and understand their ultimate effects.  It asks us to better understand how seemingly separate systems interact and how changes in one creates ripple effects in others.  Besides allowing us to mediate between counter-productive arguments, systems thinking also provides an opportunity to discover new solutions.

By broadening our thinking, systems thinking allows us to uncover new solutions to old problems.  If we see how changes in one system can ripple into others, we can harness these ripples for positive change in our society.  It asks us to look at why things happen, at root causes, rather than addressing the ramifications or symptoms of those problems.  It allows us to explore how numerous problems in our society may be linked by ripple effects of similar issues we haven’t imagined.  For instance, could the rising costs of US health care (and its effects on treatment of mental health issues) be a progenitor of the rising threats of violence and recruitment of disaffected youth by terrorist organizations?   Could the antiquated US tax system be a progenitor of immigration challenges, job-loss through outsourcing, and increased income divisions?  Could US foreign policy be a bigger source of terrorist threats than religious extremism?  Systems thinking helps us see how solving one challenge may also have positive benefits on others.

Unfortunately, we do not look at problems as components in a unified system of systems, we tend to look at individual problems and argue solutions without thinking about the ramifications of those arguments.  We frequently miss the forest for the trees.  The effect is to leave us in a perpetual state of uncertainty, never moving society fully forward no matter how many problems we try to solve.  We never address the true source of the problem, only applying patches that don’t align and don’t solve the underlying problem.  We would all do better if we took a more holistic view of the problems we face, rather than reactively addressing symptoms.

 

References

Mars, M., Bronstein, J., & Lusch, R. (2014). Organizations as ecosystems: Probing the value of a metaphor. Rotman Management, 73–77.

Tulasi, C. L., & Rao, A. R. (2012). Review on theory of constraints. International Journal of Advances in Engineering & Technology, 3(1), 334–344. http://doi.org/10.2307/25148735

 

Improving Multiple-Choice Assessments by Limiting Time

Standardized, multiple-choice assessments frequently come under fire because they test rote skills, rather than practical, real-world application.  Although this is a gross over-generalization failing to account for the cognitive-complexity the items (questions) are written to, standardized assessments are designed to evaluate what a person knows, not how well they can apply it.  If that were the end of the discussion, you could be forgiven in assuming standardized testing is poor at predicting real-world performance or differentiating between novices and more seasoned, experienced practitioners.  However, there is another component that, when added to standardized testing, can raise assessments to a higher level: time.  Time, or more precisely, control over the amount of time allowed to perform the exam, can be highly effective in differentiating between competence and non-competence.

The Science Bit

Research in the field of expertise and expert performance suggests experts not only have the capacity to know more, they also know in a way differently than non-experts; experts exhibit different mental models than novices (Feltovich, Prietula, & Ericsson, 2006).  Mental models represent how individuals organize and implement knowledge, instead of explicitly determining what that knowledge encompasses.  Novice practitioners start with mental models representing the most basic elements of the knowledge required within a domain, and their mental models gradually gain complexity and refinement as the novice gains practical experience applying those models in real world performance (Chase & Simon, 1973; Chi, Glaser, & Rees, 1982; Gogus, 2013; Insch, McIntyre, & Dawley, 2008; Schack, 2004).

While Chase and Simon (1973) first theorized that the way experts chunk and sequence information mediated their superior performance, Feltovich et al. (2006) suggested these changes facilitated experts processing more information faster and with less cognitive effort contributing to greater performance. Feltovich et al. (2006) noted this effect as one of the best-established characteristics of expertise and demonstrated in numerous knowledge domains including chess, bridge, electronics, physics problem solving, and medical applications.

For example, Chi et al. (1982) determined that the way novices and experts approach problem-solving in advanced physics was significantly different despite all subjects having the same actual knowledge necessary for the problem solution; novices focused on surface details while experts approached problems from a deeper, theoretical perspective.  Chi et al. also demonstrated the novice’s lack of experience and practical application contributed to errors in problem analysis requiring more time and effort to overcome. While the base knowledge of experts and novices may not differ significantly, experts appear to approach problem solving from a differentiated perspective allowing them more success in applying correct solutions the first time and recovering faster when initial solutions fail.

In that vein of thought, Gogus (2013) demonstrated that expert models were highly interconnected and complex in nature, representing how experience allowed experts the application of greater amounts of knowledge in problem solving.  The ability for applying existing knowledge with greater efficiency augments the difference in problem-solving strategy demonstrated by Chi et al. (1982).  Whereas novices apply problem-solving approaches linearly one at a time, experts evaluate multiple approaches simultaneously in determining the most appropriate course of action.

Achieving expertise is, therefore, not simply a matter of accumulating knowledge and skills, but a complex transformation of the way experts implement that knowledge and skill (Feltovich et al., 2006). This distinction provides clues into better implementing assessments to differentiate between expert and novice: the time it takes to complete an assessment.

Cool Real-World Example Using Football (Sorry. Soccer)

In an interesting twist on typical mental model assessment studies, Lex, Essig, Knoblauch, and Schack (2015) asked novice and experienced soccer players to quickly and accurately decide the best choice of tactics (either “a” or “b”) given a video image of a simulated game situation.  Lex et al. used eye-tracking systems to measure how the participants reviewed the image, as well as measuring their accuracy and response time.  As one would expect, the more experienced players were both more accurate in their responses, as well as quicker. Somewhat surprising was the reason experienced players performed faster.

While Lex et al. (2015) determined both sets of players fixated on individual pixels in the image for nearly the same amount of time, experienced players had less fixations and observed less pixels overall.   Less experienced players needed to review more of the image before deciding, and were still more likely to make incorrect decisions.  On the other hand, more experienced players, although not perfect, made more accurate decisions based on less information.  The difference in performance was not attributable to differences in basic understanding of tactics or playing soccer, but the ability of experienced players to make better decisions with less information and taking less time.

The Takeaway

Multiple-choice, standardized assessments are principally designed to differentiate what people know, with limited ability to differentiate how well they can apply that knowledge in the real world.  Yet, it is also well-established that competent performers have numerous advantages leading to better performance in less time.    If time constraints are actively and responsibly constructed as an integral component of these assessments, they may well achieve better predictive performance; they could do a much better job of evaluating not just what someone knows, but how well they can apply it.

 

References

Chase, W. G., & Simon, H. A. (1973). The mind’s eye in chess. In Visual Information Processing (pp. 215–281). New York, NY: Academic Press, Inc. http://doi.org/10.1016/B978-0-12-170150-5.50011-1

Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1, pp. 7–75). Hillsdale: Lawrence Erlbaum Associates.

Feltovich, P. J., Prietula, M. J., & Ericsson, K. A. (2006). Studies of expertise from psychological perspectives. In The Cambridge handbook of expertise and expert …. New York, NY: Cambridge University Press.

Gogus, A. (2013). Evaluating mental models in mathematics: A comparison of methods. Educational Technology Research and Development, 61(2), 171–195. http://doi.org/10.1007/s11423-012-9281-2

Insch, G. S., McIntyre, N., & Dawley, D. (2008). Tacit Knowledge: A Refinement and Empirical Test of the Academic Tacit Knowledge Scale. The Journal of Psychology, 142(6), 561–579. http://doi.org/10.3200/jrlp.142.6.561-580

Lex, H., Essig, K., Knoblauch, A., & Schack, T. (2015). Cognitive Representations and Cognitive Processing of Team-Specific Tactics in Soccer. PLoS ONE, 10(2), 1–19. http://doi.org/10.1371/journal.pone.0118219

Schack, T. (2004). Knowledge and performance in action. Journal of Knowledge Management, 8(4), 38–53. http://doi.org/10.1108/13673270410548478

Diversity is not just a social issue, it is an economic one.

The problem with treating diversity as only a social-justice issue is that social issues rarely get solved without demonstrating how they indirectly affect all people, not just the disenfranchised.  All you must do is look at the history of social corporate responsibility (CSR) to see this effect (O’Toole & Vogel, 2011).  CSR was treated with mostly lip-service until two things were demonstrably clear in the marketplace: 1) consumer trends were changing to favor organizations demonstrating CSR principles; and, 2) CSR (or sustainable) practices made economic sense by reducing waste, improving operations, and elevating brand.  While the messaging is about being socially conscious, CSR business models can lead to competitive differentiation, which leading to profits. The only real academic argument against CSR is the implied altruistic nature of most CSR proponents; every corporation engaging in CSR generates either direct or indirect economic profit from those actions, meaning CSR is nothing more than “enlightened self-interest” (Smith, 2003).  As much as we should care about social issues, until they affect us directly, critical mass is not achieved towards solving them.

Diversity is no different.  We, as a collective species, should promote diversity (religious, nationality, sex, age, values, etc.) simply because it is ethical and just. However, because of the numerous permutations of bias, no single group of disenfranchised gains sufficient support to make true change solely on the basis of justice.   Unless we can demonstrate how biases affect everyone, progress will remain slow, or non-existent.  The best way to combat isses of diversity is through developing the enlightened self-interest of the greater society.

Fortunately, there is tremendous support that diversity is the key to economic profit and productivity benefiting everyone, including those who are not victims of bias.  It is no accident that our national headlines, business articles, and social commentary are inundated with stories about both diversity as well as innovation.  These two concepts are intimately conjoined. Without diversity of experience, thought, and perspective, innovation does not happen; without innovation, society will no longer grow and prosper, but decline.  This affects everyone.

Innovation is the only true competitive differentiator in today’s world economy (Drucker, 1992; Friedman, 2006; Salchow Jr., 2016; Teece, 1998, 2004). Whether the innovation is a means to increase organizational efficiency, develop new business models, or an innovative product, the days of competing solely on accumulated land, capital, equipment, or market dominance are long over.  Those who don’t innovate, fail in the long-run. This affects people, companies, communities, and countries not just certain individuals. The inability to adapt to the global economy has decimated entire regions in the U.S. from miners, to steal producers, to manufacturers.   The number of companies and individuals directly affected is miniscule to the number of companies, individuals, and communities that have collapsed indirectly from these failures.  Lack of innovation capability affects us all.

Yet, we know that diversity in perspective, knowledge, experience, and capabilities is a foundation of innovation (Gladis, 2017). We know that diversity drives innovation (Niebuhr, 2010; Parrotta, Pozzoli, & Pytlikova, 2014), and creates economic rents, productivity, and success (Beck & Walmsley, 2012; Crook, Todd, Combs, Woehr, & Ketchen, 2011; Kim & Ployhart, 2014). Without diversity, we cannot hope to innovate because innovation is all about seeing things from a different perspective, a different value structure, a different life experience, a different cognitive lens.  It is through exploring and evaluating these differences that we see new possibilities, new solutions, and new ways of moving forward as companies, communities, and societies.  Diversity forces us to challenge what we think we know, and that leads to innovation.

It is sad that at a time when collaboration and access to diverse perspectives is so easy, we have instead taken to divisiveness, to segregation. We seek the illusionary safety of the known and miss the forest for the trees that don’t look, act, talk, or believe like us.  However, if we fail to see how diversity is an asset, not a liability, we fail our society. We fail, not because we violate the social contract, but because we will bankrupt society.  We fail by succumbing to what we believe is, rather than seeing what could be.  Without innovation, driven by diversity, we become static and eventually decline (Second Law of Thermodynamics anyone?).

Diversity is an economic imperative, not just a social one. The best way to secure your own future, is to seek out and embrace diversity. It is in our own self-interest.

References

Beck, J. W., & Walmsley, P. T. (2012). Selection ratio and employee retention as antecedents of competitive advantage. Industrial and Organizational Psychology, 5(1), 92–95. http://doi.org/10.1111/j.1754-9434.2011.01410.x

Crook, T. R., Todd, S. Y., Combs, J. G., Woehr, D. J., & Ketchen, D. J. J. (2011). Does human capital matter? A meta-analysis of the relationship between human capital and firm performance. Journal of Applied Psychology, 96(3), 443–456. http://doi.org/10.1037/a0022147

Drucker, P. F. (1992). The post-capitalist world. Public Interest, 109(Fall 1992), 89–101. Retrieved from http://www.nationalaffairs.com/

Friedman, T. L. (2006). The world is flat: A brief history of the twenty-first century. New York, NY: Farrar, Straus and Giroux.

Gladis, S. (2017). The Formula for Achieving Innovation. TD: Talent Development, (February).

Kim, Y., & Ployhart, R. E. (2014). The effects of staffing and training on firm productivity and profit growth before, during, and after the Great Recession. The Journal of Applied Psychology, 99(3), 361–89. http://doi.org/10.1037/a0035408

Niebuhr, A. (2010). Migration and innovation: Does cultural diversity matter for regional R&D activity? Papers in Regional Science, 89(3), 563–585. http://doi.org/10.1111/j.1435-5957.2009.00271.x

O’Toole, J., & Vogel, D. (2011). Two and a half cheers for conscious capitalism. California Management Review, 53(3), 60–76. http://doi.org/10.1525/cmr.2011.53.3.60

Parrotta, P., Pozzoli, D., & Pytlikova, M. (2014). The nexus between labor diversity and firm’s innovation. Journal of Population Economics, 27(2), 303–364. http://doi.org/10.1007/s00148-013-0491-7

Smith, H. J. (2003). The shareholders vs. stakeholders debate. MIT Sloan Management Review, 44(4), 85–90. Retrieved from http://sloanreview.mit.edu/

Teece, D. J. (1998). Capturing value from knowledge assets: The new economy, markets for know-how, and intangible assets. California Management Review, 40(3), 55–79. http://doi.org/10.2307/41165943

Teece, D. J. (2004). Knowledge and competence as strategic assets. Handbook on Knowledge Management 1: Knowledge Matters, 40(3), 129–152. http://doi.org/http://dx.doi.org/10.1007/978-3-540-24746-3_7

Misconceptions about Certification

There seems to be wide misconceptions about what “Certification” and “Licensure” are all about.  Some see it as just the final part of an educational regimen.  Other’s see it as some kind of hurdle imposed by greedy organizations restricting access to some benefit.  These misconceptions and skewed perspectives, lead them to make demands affecting the very heart of what certification is all about and minimizing the value of the very certification they are working to achieve. First:

What Certification IS …

Certification is a legally defined classification stating a certifying body stands behind the capability of a certified individual to perform at some specific level of capability.  Certification is a very simple construct: it is the definition of a standard of performance, and a program assessing individuals to that standard.  That’s it, nothing more and nothing less.

Setting a standard is the first part of any certification.  Ideally, this standard is defined with help of people who perform the job.  In addition, a standard implies that everyone, no matter whether they have done the actual job for decades or participated in the development of the standard itself, must objectively demonstrate their capability in exactly the same manner.  Unless this standard is objectively applied equally to all individuals, it is not a standard.  Maintaining that standard is the foundation of certification value; it is what the certification stands for and how it should be used.

Creating an assessment of that standard is the second part of any certification.  This not only includes a mechanism assessing current capability, but also a program or process to ensure future capability.   Despite what many believe, creating an assessment is not a simple, ad hoc process where someone creates an assessment (test) and makes people take it to prove their ability.  It is, in fact, a highly rigours process backed by decades of scientific research on measuring cognitive ability (psychometrics), the sole purpose of which is ensuring the validity of the decisions made based on assessment performance.  It involves designing the assessment, job-task analysis of the job being assessed, formalized standards of item (question) writing, public beta-testing of items, psychometric evaluation of the item performance, assessment construction, and evaluation of appropriate scoring.  Done properly, it is time-consuming and costly (which is why some organizations don’t do it properly).

Finally, because knowledge and competence are perishable commodities, mechanisms must be put in place ensuring certified individuals remain capable in the future.  This is frequently done by limiting the length of time a certification is valid and requiring periodic re-certification (re-validation) of the individuals capability.  Other methods may include proving continued education and practice of the knowledge.  Regardless, the ongoing evaluation of certified individuals must adhere to the original standard with the same validity, or the standard no longer has value.

There is no hidden agenda to certification.  There is no conspiracy.  It is simply to establish a standard and assess individuals compared to that standard.

Misconceptions about Certification

Really, any belief beyond the design of a standard and the assessment to that standard, is a misconception about certification.  However, there are a number of misconceptions that often drive changes detrimental to the rationale of certification.  The most common ones relate to understanding what an assessment is for, training, and re-certification requirements.

Most people mistakenly assume the items within an assessment must represent the end-all-be-all of what someone should know.  They don’t understand that a psychometric assessment is not about the answers themselves, but the inferences we can make about performance based on those answers.  There is no way to develop an exhaustive exam of all the knowledge necessary to be competent and to deliver that exam efficiently.  However, we can survey a person’s knowledge and through statistical analysis infer whether they have all the knowledge necessary or not.  The answer to any specific question is less important than how answering that question correlates with competence.  Even a poorly written, incomplete, and inaccurate item can give us information about real world performance; in fact, evaluating how a candidate responds to such an item can be highly informative (although this is not a standard, intentional practice).  This focus on the items themselves, rather than the knowledge and competence the answers suggest, is what makes people incorrectly question the validity of the certification.

Similarly, many people think a certification should be able to be specifically taught.  As such, they believe a training course should be all that is necessary to achieve a certification.  However, this does not align with what we know about the development of human competence.  There is a big difference between knowing something, and knowing how to apply that knowledge competently.  Certification is an assessment of performance, not knowledge; and, as such, cannot be taught directly.  If someone can take a class and immediately achieve certification, either: A) the assessment does not evaluate actual performance; or, B) the course simply teaches the answers to the questions on the exam, rather than the full domain of knowledge.  In either case, you have biased the inferences made by the assessment.  Competence begins with knowledge, but must also have experience and practice.  This cannot be gained through a class, but only through concerted effort; you cannot buy competence.

Finally, many people also believe that once a certification has been achieved, it shouldn’t need to ever be evaluated again; or, that taking a course instead of an assessment should suffice.  The former belief simply ignores the fact performance capability is a perishable commodity: if you don’t use it, you lose it.  The latter once again confuses knowledge with performance.  How frequently this needs to happen, or whether continued education is sufficient to demonstrate continued performance is entirely dependent on the knowledge domain the certification attempts to assess.  In highly dynamic environments, this may need to be done much more frequently and rely more on assessments than in other domains; however, ongoing evidence of continued capability is a must if standards are to be maintained.

Leave Certification to the Professionals

The heart of the problem is that everyone seems to believe they are experts in the design of certification programs and assessments simply because they have participated in them.  The reality is that certification is a rigorous, research-based, and scientific endeavor.  The minimum requirement to be considered a psychometrician is a PhD; that’s a great deal of specialized knowledge most people do not have.  The decisions made are not arbitrary, nor are they made with the intention of anything other than maintaining the standard and making valid assessments of individuals according to that standard.

At the end of the day, the value of a certification is whether the people who achieve it can perform according to the standard the certification set forth.  If the certification cannot guarantee that, then it is not valid and has no value.  However, this requires people to actually understand what that standard is, what it means, and why it was created.   It requires people to accept there is a rigorous process accounting for all of the decisions and those decisions all support validity.  Finally, it requires people to understand that just because they may be experts in their field, they are not experts in certification.