Interest and Skill measurement in parallel – the Campbell Interest and Skill Survey 

N. Taylor & C. Donnelly

Introduction

The Campbell Interest and Skill Survey (CISS) has become a well-known instrument in the career interest assessment arena of vocational psychology.   The instrument has some unique features that can be beneficial in certain uses and populations in South   Africa.  This chapter aims to provide the reader with an objective evaluation of the instrument, particularly from a South Africa perspective.  The chapter takes the reader on the instrument’s development journey, its technical properties and the benefits and limitations of the use of the instrument.

Development History of the CISS

The CISS has development roots stretching back a number of decades.  David P. Campbell, the primary developer of the CISS, worked on the development and revision of another well-known interest instrument, the Strong Interest Inventory (SII).  The SII still remains one the most used interest inventories (Hansen & Campbell, 1985, Walsh & Betz, 1995, Zytowski & Warman, 1982; as cited in Kaplan & Saccuzzo, 2001).

Campbell entered the interest assessment area during the 1950’s at the University of Minnesota.  His entry was mostly due to E.K. Strong’s illness at the time and become closely involved in the research and development of an earlier version of the SII, the Strong Vocational Interest Blank (SVIB).  Through a number of changes to the SVIB, Campbell then published a newly revised instrument in 1974 called the Strong-Campbell Interest Inventory (SCII).  The new revision included the development of a single-sex form as opposed to the previous version (Campbell, 1995).    Pressure from the gender equality movement created pressure on interest instrument publishers to ensure fair cross-gender assessment of interests.  The argument was that interest instrument tended to recommend gender stereotyped occupations.

Later on, in the 1980s, the SCII was modified in a number of significant ways, mostly due to a new researcher, Hansen (Campbell & Hansen, 1981; Hansen & Campbell, 1985).  The improvements were: (a) a more balanced gender composition of the general reference sample, (b) an expansion of the profile coverage to include more blue-collar occupations, (c) a concerted effort to provide both female and male scales for almost all of the occupations, and (d) an increase in the average size of the occupational samples (Campbell & Hansen, 1981; Hansen & Campbell, 1985).  Unfortunately, between 1983 and 1988, a legal battle ensued between Campbell and Stanford University Press regarding the intellectual property rights of the SCII. The outcome of the legal confrontation was that Stanford gained all rights to the inventory and renamed it the Strong Interest Inventory (SII).  However, Campbell retained the rights to use his name for the purposes of instrument development as it was no longer tied to the SII (Campbell, 1995).   Campbell had accumulated extensive knowledge years of test development.  Later, Campbell then initiated the development of the CISS (Campbell, 1995, p. 392).

The CISS was published in 1992 (Campbell, Hyne & Nilsen, 1992; Campbell, 1993; Campbell, 1995).  The CISS expands on earlier work of Campbell by adopting a more appropriate item pool[1] and a more flexible six-point Likert-type item response format[2].  In addition to the changes to the item pool, a new skills measurement model was added in parallel to interests – the first of its kind at the time. The skill measurement reflects an individual’s self-assessed level of skill in a variety of activities. The authors felt that this type of skills measurement, even though not objective, may provide some idea of the person’s propensity for having a certain skills set which could be valuable in making career decisions (Campbell et al., 1992). All of the scales were standardised on the same population and both genders are scored and norm-referenced in the same way.  Both genders are compared to one combined gender norm. It is an instrument that has the advantage of benefitting from years of test development knowledge and modern technology (Campbell, 1995).

The questionnaire focuses on careers that require tertiary education.  Therefore, it would be most appropriate to assess the interest profiles of individuals intending to enrol at university, are currently at university or have completed university.  The instrument is also useful when adults wish to make career transitions or wish to understand specific current job dissatisfactions (Boggs, 1999).  The American reading level is at the sixth grade and the instrument is therefore, in terms of reading proficiency, appropriate for adults and adolescents aged fifteen and older.  It has, however, been used at younger ages in exceptional circumstances (Campbell et al., 1992).  An objective assessment of the South African reading level has not been conducted (N. Taylor, personal communication, 2009), however, the local test distributor recommends grade 12 English proficiency.

Individuals who are assessed on this instrument are required to evaluate their own levels of interest on 200 academic and occupationally oriented items (85 occupations, 43 school subjects and 72 activities).  The questionnaire also requires that individuals assess their own level of skill in 120 items based on occupational activities (Campbell et al., 1992).

Scales and Interpretation of the CISS

The CISS provides the following measurement scales for the individual taking the assessment: Basic Interest and Skill scales, Orientation scales, Occupational scales, Special scales and Procedural checks.

Basic scales

The Basic scales provided the foundation of the CISS development and measurement model (Campbell et al., 1992). Hence, the Basic scales reflect the first-order factors measured by the CISS. The Basic scales, in turn, load onto seven second-order/global factors measured by the Orientation scales.   The Occupational scales are criterion-keyed scales where mean scores have been calculated for a given occupation – the respondent’s results are then compared with the criterion group to determine whether their results are similar to those happily employed in the given occupation.

During the early stages of the development of the CISS, different, non-matching sets of interest and skill scales had been developed, working from item intercorrelations. The construction of the respective scales was based on the approach taken with the Strong series of questionnaires; that is, by examining the item intercorrelations to determine clusters that could be representative of unique interest/skill areas.  The finding, however, was that it is virtually impossible for respondents to understand the interest and skill scales as functioning with different first-order factor structures (Campbell et al., 1992). Therefore, the interest and skill measurement models are viewed as being measured in a parallel fashion.

Table 1 depicts the manner in which the first-order interest and skill factors load onto the second-order orientation factors and at the same time defines the Basic scales in terms of the core activities that denote the latent interest and skill dimensions measured by the Basic scales.

The activities listed next to each first-order factor should likewise be interpreted consecutively as activities one would like to perform and one would feel confident to perform if the interest and skill dimension would be strongly developed.

Orientation scales

The authors of the CISS constructed seven Orientation scales for both the interest and skill measurement.  The Orientation scales serve as global factors which summarise the information gained through the primary Basic scales.  The final CISS report presented to the client initially shows information at the global level, then proceeds to provide more detailed information at the primary level (Basic scales) and finally provides information derived from the Basic scales (Occupational and Special scales) (Campbell et al., 1992).

Although the interest orientations measured by the Orientation scales constitute the major organising structure of the CISS, the Basic scales (as discussed in the previous section) were constructed prior to the Orientation scales.  In constructing the Orientation model, the developers’ primary goal was to create between five and seven broad categories of interests in terms of which the Basic Interest and the Basic Skill scales could be summarised (Campbell et al., 1992).

Principle component analyses were conducted on the Basic interest scales.  The appropriateness of a five, six and seven-component solution was considered.  Based on the Kaisers criterion, inspection of the scree plots and the factor interpretability of the various solutions, the authors felt that the seven-component solution summarised the data best (Campbell et al., 1992).  The seven interest components were defined above as the seven interest orientations.  The seven interest orientations therefore, essentially, are seven second-order interest factors.  This was also investigated for the Basic skill scales, and a final six factor structure was found, but the authors felt that it is not possible to have one factor structure at a global level for interests and not have the same for skills, therefore a seven factor structure is provided for both interest and skill (Campbell et al., 1992).

The instrument’s technical manual provides information regarding the correlations[3] between the Basic scales and the seven Orientations.  In Campbell et al.’s (1992) opinion, this approach best covers the entire breadth of their domains.  Mean correlations between Basic Interest scales and Orientations vary between 0.60 and 0.90 with a median of 0.72.  Mean correlations between Basic Skill scales and Orientations vary between 0.65 and 0.90 with a median of 0.77.

The following represents the seven broad (global level) areas of interest and self-reported skill include[4]:

i) Influencing – covers the general area of leading and influencing others; influencing others through leadership, politics, public speaking and marketing.  Typical high-scoring occupations include company presidents, corporate managers and school superintendents.

ii) Organizing – activities that bring orderliness and planfulness to the work environment. Organizing the work of others, managing and monitoring financial performance.  Typical high-scoring occupations include accountants, financial planners, office managers and administrative assistants.

iii) Helping – involves helping and developing others; helping others through teaching, healing and counselling.  Typical high-scoring occupations include counsellors, teachers and religious leaders.

iv) Creating – includes artistic, literary and musical activities; creating artistic, literary or musical productions, and designing products or environments.  High-scorers include artists, musicians, designers and writers.

v) aNalyzing – involves scientific, mathematical and statistical activities; analyzing data using mathematics and carrying out experiments. Typical high-scorers are scientists, medical researchers and statisticians.  This interest orientation is labelled with the letter N as A is used for the Adventuring orientation (see below).

vi) Producing – covers practical, hands-on, productive activities. Producing products using “hands-on” skills in farming, construction and mechanical crafts.  Typical high scoring occupations include: mechanics, veterinarians and landscape architects.

vii) Adventuring – includes activities in athletics, the police and military. Adventuring, competing and risk taking. Typical occupations: military officers, police officers and athletic coaches.

The relationship between the seven orientations is represented in Figure 1.  Orientations closer to each other are regarded to be similar in the sense that individuals that would be attracted to – and would feel confident in a specific area – would tend to also be attracted to and feel confident in interest areas in close proximity to the focal area, albeit somewhat less so.  Those orientations that are diagonally across are regarded to be dissimilar in that individuals that would be attracted to and would feel confident in a specific area would tend to not be attracted to and feel confident in interest areas diagonally opposite the focal area.  This is akin to the hexagonal relationships of interests as proposed by Holland (1985), which is operationalised through the Vocational Preference Inventory and many other forms of this interest model.

Although similar to the Holland typology an additional category, Adventuring, has been added.  The similarities and differences between the Holland typology and the CISS are represented in Table 2.  The differences include: (a) the CISS Influencing orientation focuses more on leadership, whereas the Holland Enterprising type is tilted toward sales activities, (b) The CISS Organizing orientation focuses on management and financial services, whereas Holland’s Conventional type looks at office and clerical work; (c) The CISS presents the  Holland’s Realistic type as two orientations, namely Producing and Adventuring.

Norming of the Orientation scales was conducted by gathering the data from 65 occupational samples.  A raw score mean was calculated on each Orientation scale for female respondents (n=1790) as well as for male respondents (n=3435).  The two raw score means were averaged to create the effect of equal gender weighting; the unweighted means of means is then used in the raw-score-to-standard-score conversion formula. The authors aimed to ensure that the standard deviations of all the scales are approximately equal across gender groups.    Ultimately, respondents’ Orientation scale raw scores are compared to happily[5] employed people spread over 65 equally represented occupational samples (Campbell et al., 1992).

The above norming process was also applied in generating norms for the Basic scales.  No norms have as yet been created for the South African context (N. Taylor, personal communication, 31 October, 2008).

As with the Orientation scales, higher scores in the Basic scales indicate an attraction for the activities that the Basic scales measure.  High scores on the Basic skill scales represent a self-perceived confidence in the activities that the Basic scales measure (Campbell et al., 1992).

Occupational scales

The Occupational scales of the CISS provide the individual seeking their ideal career with additional information regarding their interest in relation to other happy individuals in their chosen careers (Campbell et al., 1992). This is achieved through a derived set of interests and skills that match with a particular occupation.  This is essentially the criterion referencing technique that Strong pioneered in his work on interest assessment.  “They [Occupational scales] were developed empirically by contrasting the responses of occupational samples with those of a reference sample widely drawn from the working world” (Campbell et al., 1992, p. 138).

The Occupational scales assist in providing the career finder with occupational titles that may warrant further investigation because they suit his/her specific profile of occupational activity interests and skills.  This scale also has the benefit of turning the assessment into commonly understood terms in contrast to the more abstract conceptual terms used in the Orientation and Basic scales.

Special scales

The CISS contains three additional special scales that will further assist the career explorer.  The three scales are: (a) an Academic Focus scale, (b) an Extraversion scale and (c) a Variety scale.  As with the Occupational scales, these scales were derived by identifying items from the measurement model and comparing the interests and self-reported skills to individuals’ scores at various levels in terms of academic, extraversion and need for variety preferences.

The Academic Focus scales measure the respondent’s interest and confidence in doing well in academic settings.  Scores on the Extraversion scale reflect the strength of the respondent’s interest and self-confidence in working with people.  The Variety scale indicates an individual’s preference for diverse interests/skills – however the authors are not clear on how this score should be used (Campbell et al., 1992).  The authors (Campbell et al., 1992) do however, point out that the variety scale score does seem to indicate whether respondents opt for the extreme positive statements on many items.

Procedural Checks

Procedural checks allow the interpreter of the results an opportunity to determine whether the results are valid and can be used.

Response Percentage Check

This procedural check assists in detecting problems that may have occurred during administration, completion and processing of the answer sheets.  The instruments answer sheet is divided in to sections, this procedure check provides an indication of how many items have been completed (Campbell et al., 1992).

Inconsistency Check

The inconsistency check was constructed by assembling 10 pairs of extremely similar items and then comparing the individual’s responses to each pair of items.  If a certain number of paired items have inconsistent results this would be interpreted that the profile could be invalid (Campbell et al., 1992).

Technical Properties of the CISS

Official Psychometric Properties of the CISS: Technical Manual

The technical aspects of the instrument are contained in the following discussion.  The information provided by the technical manual is present first.  It would be important to note that the information from the technical manual was derived from an American standardisation sample; it does not provide South African data.  This is provided later in the section.  While the technical manual does provide positive information regarding the instrument, some detail regarding independent international research is provided.

Orientation scales

Reliability.

Reliabilities reported in the CISS test manual (Campbell et al., 1992) focus on internal consistency reliability coefficients (alpha) and test-retest correlations.  At the time of publishing satisfactory internal consistency was established.  All scale reliabilities alpha values were above the rule of thumb level of 0.70.  However, in general, reliability levels were higher.  Test-retest reliability was also investigated and presented in the manual, and results also suggested good temporal stability for the instrument.  Studies revealed test-retest reliability coefficients above 0.80.

Validity.

Validity of the CISS, as reported in the manual (Campbell et al., 1992), is represented only by scale intercorrelations and by examining the scores of individuals engaged in occupations that would theoretically attract a score higher on specific scales.  In the latter instance the interest and skill scores of individuals in occupations in which the specific interest and skills are theoretically expected to be related to satisfaction and success were correlated with these two criterion variables. If the relevant interest and skill distributions would differ significantly in terms of the mean across  contrast groups created in the occupational sample in terms of their degree of engagement, it would mean that the interest and skill inferences derived from the scales are valid.

The authors of the CISS evaluated the construct validity by calculating correlations between the Interest Orientation scales and the Skill Orientation scales (in essence convergent validity[6]) and they report correlations between 0.76 and 0.66 with a median of 0.70.  This means that approximately 50% of their variance is in common (Campbell et al., 1992).  This is suggestive of simplistic construct validity.  Construct validity should also be conducted by comparing the CISS constructs with a well-known, validated instrument that measures the constructs that the CISS aims to measure.  This would be a more objective approach.  An alternative approach would be to conduct confirmatory factor analysis to determine whether the constituted constructs and their associated relationships presents as per the defined model in the data. South African research has been conducted in this respect.

Concurrent and predictive validity was established by calculating mean scores for both interest and skill scales for 58 different occupational samples.  The mean scores were then ranked from highest to lowest to show which types of occupations are occupied by people with strong interests and confidence in each Orientation scale.  The top five occupations for each Orientation are as follows (Campbell et al., 1992):

i.)            Influencing: Media Executive, Marketing Director, Hotel Manager, Public Relations Director and Manufacturer’s Representative.

ii.)           Organizing: Accountant, Bank Manager, Retail Store Manager, School Superintendent and Bookkeeper.

iii.)         Helping: Religious Leader, Guidance Counsellor, Child Care Worker, Nursing Administrator and Athletic Trainer.

iv.)         Creating: Fashion Designer, Liberal Arts Professor, Translator/Interpreter, Librarian and Commercial Artist.

v.)           Analyzing: Maths/Science Teacher, Medical Researcher, Chemist, Statistician and Veterinarian.

vi.)         Producing: Veterinarian, Agribusiness Manager, Carpenter, Landscape Architect and Test Pilot.

vii.)        Adventuring: Police Officer, Athletic Coach, Military Officer, Test Pilot and Athletic Trainer.

Overall, it would seem as if the content of each Orientation scale relates to occupations fitting of these orientations.  The evidence suggests that people tend to migrate to and remain in occupations that correspond to their interest in activities associated with those occupations as well as their confidence in performing activities related to those occupations. This provides some, albeit limited, support for the concurrent validity of the Orientation scales.  Stronger, more convincing evidence of the criterion-related validity of the career-related inferences derived from the CISS Orientation scales would be obtained if appropriate criterion variables (for example: satisfaction and career success) would be regressed on linear composites of interest and skill scales that would theoretically be expected to explain variance in the criterion variable.

Basic scales

Reliability.

Reliabilities for the Basic scales were of the levels seen for the Orientation scales.  All were satisfactory and suggest good internal consistency and temporal stability.

Validity.

The nature and magnitude of the scale intercorrelations (construct validity) and by examining occupational samples that score higher on specific scales that should, theoretically, link to the said occupations (criterion validity) was the approach taken for the Basic scales construct validity. If this is found then a true positive or “good hit” would be indicated (i.e., those scoring higher on scales that match are employed in related occupations).

The authors evaluated the construct validity of the Basic scales by calculating correlations between the Basic Interest scales and the Basic Skill scales (in essence convergent validity) and report correlations between 0.80 and 0.46 with a median correlation of 0.68 (Campbell et al., 1992).  Again it needs to be noted that the CISS manual does not provide sufficient psychometric evidence to allow a confident positive verdict on the construct validity of the instrument, and additional validity studies are reported in the following sections.

Concurrent and predictive validity was established by calculating mean scores for both interest and skill scales for 58 different occupational samples.  The mean scores were then ranked from highest to lowest to show which types of occupations are occupied by people with strong interests and confidence in each of the Basic scales (Campbell et al., 1992).  Examination of the tables in the CISS test manual (Campbell et al., 1992) reveals that the distribution of occupations across the Basic scale are generally the same as was found for the Orientation scales.  Regression analyses in order to predict occupation choice (as well as additional relevant career criterion variables like career satisfaction and career choice) as a result of reported interest and skill were not conducted.  This is unfortunate, as it would have certainly made for a stronger argument for the concurrent and predictive validity (depending on the validation design) of the career-related inferences typically derived from the CISS.

Psychometric Properties of the CISS: Independent International Research

Construct validity of the interest scales

Sullivan and Hansen (2004) conducted research focussing on the construct validity of the interest scales of the CISS.  The study looked at all three scales of the CISS, Orientation, Basic and Occupational Interest scales.  The correlational study compared CISS scales with the SII scales.  In some cases it was not easy to identify the corresponding scale on the SII. Nevertheless the analysis on the matching scales was conducted.  The study also aimed to evaluate the hexagonal relationship of the Orientation scales as suggested by Campbell (Campbell et al., 1992) as a reflection of the Holland model.

The results of the analyses indicate that for the Orientation scales and the matched scales (in the SII) the median correlation for females was 0.72 whereas the median correlation for non-matching scales was 0.17.   Similar findings for the men are a median correlation of 0.69 for matching scales and 0.15 for non-matching scales.   However, the Adventuring scale demonstrated the least evidence of convergent validity which is not surprising considering that this scale is not directly assessed by any other interest measure. Overall, the matching Orientation scales share about 50% of the variance, and non-matching scales share about 2% (Sullivan & Hansen, 2004).  Findings in this study also support the hexagonal relationship.   Therefore independent research does seem to demonstrate some level of objective construct validity.

The CISS Basic scales were matched with the SII Basic Iinterest Scales as was the case with the Orientation scales analyses. The median correlation for women was 0.75.  The median correlation between non-matching scales was 0.15. For men, the matching median correlation was 0.74 and the non-matching median correlation 0.15.   This again suggests evidence of good construct validity.

Factor analyses were also conducted using both the SII Basic Interest Scales and CISS Basic scales to determine which scales from both instruments share variance.  The purpose of this analysis was to confirm construct validity by using the well-established SII.  Some gender differences were noted. For women the Investigative scales (CISS: Analyzing; SII: Investigative) and two of the Realistic scales (CISS: Producing; SII: Realistic) loaded on a single factor. The CISS Adventuring scale did not load onto this factor.  This suggests that females may not distinguish between these two categories of interest.  However for females, the Artistic scales loaded on a different factor than the Social scales.  Interestingly enough, for females the Adventuring scales loaded with the Social scales.  Involvement of females in adventure activities seem to be motivated by a social interaction (as per the SII) need.  This scale should be more closely aligned with the Realistic scales.  For men, the Social (CISS: Helping; SII: Social) and Artistic (CISS: Creating; SII: Artistic) scales loaded on a single factor, suggesting that men do not distinguish between these two categories of interest (Sullivan & Hansen, 2004).

As for the Occupational scales, the median correlation for matching scales was 0.62 for women and 0.66 for men.  Non-matching scales indicate a median correlation of 0.06 for woman and 0.05 for men.  Again, this provided evidence of construct validity for the Occupational scales.

Construct validity of the skill scales

Hansen and Leuty (2007) conducted a further research study that aimed to clarify the skill construct in the CISS.  The study aimed to firstly test the convergent and divergent validity of the CISS Skill scales on the CISS Interest scales.  A second objective was to compare the CISS Skill scales and the SII Basic Interest scales to determine how the CISS Skill scales correlate with the SII Basic Interest scales. This approach was necessary to evaluate the relationship between self-reported skill and interest independent of the similar item content as seen in the CISS.  A third analysis comprised of correlating the CISS Skill and Interest scales with an independent measure of self-perceived abilities; the Minnesota Abilities Estimate Questionnaire (MAEQ, Desmond & Weiss, 1973, as cited by Hansen & Leuty, 2007).

The matching CISS Skill and Interest scales correlated between 0.46 and 0.71 for females, and between 0.62 and 0.72 for males.  For the most divergent scales correlations varied between -0.1 and 0.00.  For ease of interpretation of the Basic Interest and Skill scale correlations, the various matched Basic Skill and Interest scales were grouped into the global Orientation scales structure of the CISS and a median correlation was calculated for each Orientation. The median correlations for each group ranged between 0.22 and 0.56 for females and 0.24 and 0.55 for males. Similar findings were obtained for the Occupational scales (Hansen & Leuty, 2007). Therefore, independent research does suggest that should an individual find certain activities interesting, a fair amount of their self-perceived skills may be determined by their interest.  The converse can also be inferred.

The analyses of the CISS Orientation Skill scales and SII General Orientation Theme scales (interests) resulted in correlations ranging between 0.25 and 0.52 for similar orientation scales for females and between 0.38 and 0.59 for males (Hansen & Leuty, 2007).  Correlations were stronger for the CISS Orientation Interest scales and the SII GOT scales than for the CISS Orientation Skill scales.  This finding could be expected based on the results of the earlier Sullivan and Hansen (2004) study.  The CISS Basic Skills scales and the SII Basic Interest scales were also compared. Results indicated median correlations of between 0.19 and 0.47 for females and between 0.26 and 0.45 for males.  Similar results were found for matching Occupational Skill scales and SII Occupational Interest scales.  Based on these findings, it could be concluded that self-reported skills only share a relatively small proportion of common variance with interests and that the research evidence (Hansen & Leuty, 2007) suggests that self-reported interests in specific activities is a construct that should be distinguished from self-reported skill or confidence in those activities.  Despite a moderate correlation between the two constructs, showing interest in specific activities should be seen as conceptually distinct from being skilful or confident in displaying those activities.  Therefore, for the analyses in the current project, it is best to keep the interest and skill measurement models in the CISS separate.

The results of the third analysis: correlations between the Skills scales and the MAEQ findings reveal some significant relationships, however, the correlations are mostly modest and tend to lie between 0.02 and 0.49 for females and between 0.02 and 0.55 for males.  A limitation in this analysis is that the MAEQ measures specific self-perceived abilities, namely verbal aptitude, numerical aptitude, spatial aptitude, form perception, clerical perception, finger dexterity and manual dexterity, whereas the CISS Skill scales tend to measure interest field self-efficacy perceptions.  Therefore, it would seem as if these scales tend to measure different domains in the majority of cases. Hansen and Leuty (2007) indicate that self-perceived skill measurement may be more indicative of self-efficacy than of self-estimated abilities.

Concurrent validity for the skill scales

Two studies that focused on whether higher interest scores predicted choice of college majors were conducted by Hansen and Neuman (1999) and Pendergrass, Hansen, Neuman and Nutter (2003, as cited by Hansen & Leuty, 2007).  The results indicate that the CISS interest measurement, as well as the SII, predicts choice of college major.  The major limitation of these studies is that the CISS is recommended for both university student and working adults that are currently pursuing or have pursued postsecondary studies.  Both of these independent studies have not looked at the prediction of actual careers chosen after university. As far as concurrent prediction of the Skills scales, Hansen and Leuty (2007) conducted analyses to determine the hit rates of higher skill scores with choice of college major.  Results show that 57.7% of female students were classified as having excellent or good hits for college major selection based on their CISS Occupational Skill scales scores.  For male students, an excellent or good hit was 69%.  The differences were found to be not statistically significant.

Psychometric Properties of the CISS: Independent South African Research

South African research has only recently been conducted on the CISS (Donnelly, 2009, 2010). The results provide information regarding the reliability of the instruments as well as construct validity by making use of a confirmatory factor analytic approach.  The focus of the research was to determine whether measurement equivalence was present across gender for the CISS.  A sample of 810 individuals formed the basis of the local research conducted.  The data was gathered by the local questionnaire distributor over a period of time.  The respondents were assessed for many purposes which include: career counselling, vocational choice and personal career growth.  It may also be possible, although unlikely, that the results could have been used in a selection setting.

Basic scales

Reliability.

Reliability in the South African studies focussed on internal consistency.  For the Basic Interest scales the mean alpha was 0.83 (Donnelly, 2009).  This seemed to show consistency with the results found for the American sample.  For all scales, the alpha value exceeded the rule-of-thumb value of 0.70.   For the Basic Skill scales, the mean alpha was 0.78, again above the standard rule-of-thumb (Donnelly, 2010).  Some scales returned alphas lower than 0.70.  This was mostly due to fewer items per scale, therefore making the scales less reliable.  These results would seem to suggest suitable internal consistency reliability in the South African context.  Temporal stability has not been established in South Africa.

Validity.

Construct validly was investigated in the Donnelly (2009, 2010) studies.  The approach that was taken took the form of a confirmatory factor analytical study. Confirmatory factor analysis (CFA) assists the researcher in evaluating whether the definition and specification of a construct is consistent with the measurement of the proposed construct.  It explicitly tests the overall quality of the factor solution and the specific parameters composing a model (Kelloway, 1998).  This is achieved by constraining the data to the measurement model of the questionnaire.  If the data presents itself in a matching manner, or fits, then it is assumed that the construct is presenting itself as per the researcher’s definition of the constructs (or measurement model).   Various fit indices assist in determining how well the data fits the model.  Measurement models that fit exactly are deemed ideal – however, this is not often achieved in social science research.  Therefore closer fitting models are generally the researcher’s goal. For the CISS both the Interest and Skill Basic scales data fitted the measurement model adequately (Donnelly, 2009, 2010). In the Donnelly studies it was hoped that close fit would be achieved.  However, the results do indicate to some degree that the constructs (Basic Interest and Skill dimensions) are valid at an adequate level (Donnelly, 2009; Donnelly 2010).

Orientation Scales

Reliability.

While internal consistency reliability of the respective Basic scales have been determined (Donnelly, 2009, 2010), the Orientation scale reliabilities have not been calculated in South Africa.

Validity.

Donnelly (2009, 2010) investigated the construct validly for the Orientation scales of the CISS in South Africa.  The CFA approach was also taken to determine levels of fit between the data and the Orientation scales measurement model.  Unfortunately, poor model fit was found for both the Interest and Skill Orientation measurement models (Donnelly, 2010, Donnelly, 2009).  This would then cast doubt as to whether the constructs are as valid as intended.  However, it must be emphasised that the Orientation scales are complex factors that comprise quite a number of sub-factors.  It should be mentioned that in many cases, global (second-order) factors are not the primary level of measurement in this instrument.  Practitioners are then encouraged to place more interpretative action at the Basic scale level.  While the Orientation scales do provide an overarching summary of a respondent’s data, it certainly can not provide the level of clarity needed for an individual to make sound career decisions (Donnelly, 2009).

Interpretation and use

For ease of use and interpretation, patterns of interest and skill scores are reported on the profile as: (a) Pursue – areas that are worthy of serious consideration as the interest and skill scale scores are both high (55[7] or above); (b) Develop – seek additional training to increase self-confidence or accept as hobbies, because interest scores are high (55 or above) and skill scores are lower (54 or below); (c) Explore – gain an understanding of why the area is not more appealing or consider applying the skills to another field because interests are lower (54 or below) and skills are high (55 or above); and (d) Avoid – activities not to consider, because interests and skills are both low (45 or below).  If both interest and skill scores are in a mid-range or one is in mid-range and the other is lower, no pattern is reported (Boggs, 1999, p. 169).

In addition to the above, the test publishers have a number of tools and exercises that career counsellors can use with their clients.  These tools assist in providing further depth and discussion based on the results.  These tools are available online and can be downloaded at any time.

The CISS is available for administration in both paper-and-pencil and online versions.  Reporting includes a detailed profile chart which is now available in colour when making use of the online platform.  Access to the instrument is restricted; psychometrists and psychologists may use this tool.

Summary

While the CISS has enjoyed a long and fruitful history of use in the international field, it is not as well known in the South African market. This chapter highlighted the dearth of local research on this assessment, but also pointed to its utility in South Africa. Readers are encouraged to consider the CISS in future career applications, and also to become involved in researching the assessment further in the South African context.

References

Campbell, D., & Hansen, J. (1981). Manual for the SVIB-SCII.Palo Alto, CA: StanfordUniversity Press.

Campbell, D.P. (1995). The Campbell Interest and Skill Survey (CISS): A product of ninety years of psychometric evolution. Journal of Career Assessment, 3 (4), 391 – 410.

Campbell, D.P., Hyne, S.A., & Nilsen, D.L. (1992). Manual for the Campbell Interest and Skill Survey.Minneapolis, MN: NCS Pearson, Inc.

Donnelly, C. (2009). An investigation into the measurement invariance of the performance index. Stellenbosch: Unpublished dissertation, University of Stellenbosch.

Donnelly, C. (2010). Research report on the Campbell Interest and Skill Survey, Skill measurement model technical review.Johannesburg: Unpublished technical review document, JvR Group.

Hansen, J. C., & Leuty, M. (2007). Evidence of validity for the Skill scale scores of the Campbell Interest and Skill Survey. Journal of Vocational Behavior, 71, 23-44.

Hansen, J.C., & Neuman, J. (1999). Evidence of concurrent prediction of the Campbell Interest and Skills Survey for college major selection. Journal of Career Assessment, 7, 239-247.

Holland, J.L. (1985). Making vocational choices: A theory of vocational personalities and work environments (2^nd ed.). Englewood Cliffs, NJ: Prentice-Hall.

Kaplan, R.M., & Saccuzzo, D.P. (2001). Psychological testing: Principles, applications, and issues (5^th ed.).Belmont, CA: Wadsworth/Thomson Learning.

Kelloway, E.K. (1998). Using LISREL for structural equation modelling: A researcher’s guide.Thousand Oaks, CA: SAGE Publications, Inc.

Kerlinger, F.N., & Lee, H.B.  (2000).  Foundations of behavioral research (4th ed.).  Fort Worth, TX:  HarcourtCollege Publishers.

Sullivan B.A., & Hansen J.C. (2004). Evidence of construct validity of the interest scales on the Campbell Interest and Skill Survey.  Journal of Vocational Behavior, 66, 179-202.