Norwalk Guardian Ass'n v. Beres , 489 F. Supp. 849 ( 1980 )

RULING ON DEFENDANTS’ MOTION FOR APPROVAL OF ADMINISTRATION OF PROMOTIONAL EXAMINATION

EGINTON, District Judge.

The Norwalk Guardians Association and some of its individual members commenced this civil rights action on July 28, 1977 alleging discrimination in the hiring and promotion practices of the Norwalk Police Department. On January 10, 1978, this court (Newman, J.) entered a consent decree providing, inter alia, that

7. All promotions to be made by the Commissioners hereafter shall be made in accordance with the existing Working Agreement between the City of Norwalk and the Police Union, Local 1727, provided, however, that for a period of three (3) years from the date of this Consent Decree, the examinations for promotion and related promotion procedures shall be subject to the approval of this Court, following submission, if necessary, to counsel for the Plaintiffs, counsel for Local 1727, counsel for intervening Defendants, and the Norwalk Corporation Counsel. The panel for administering the oral examination in connection with promotions shall be increased from three (3) to five (5) persons, with a view to increasing minority representation thereon.

Pursuant to the terms of the consent decree, the city moved on October 17, 1979, for approval of a proposed examination for the rank of sergeant, which was developed by the Norwalk Police Department in conjunction with the Municipal Services Unit of the Connecticut Personnel Department (C.P.D.). Following the plaintiffs’ objection to the proposed examination, this court held an evidentiary hearing on December 18, 1979. Subsequently, all issues were briefed by the parties. On the basis of the following findings of fact and conclusions of law, this court holds that the proposed examination is content valid.

FINDINGS OF FACT

1. Promotions within the Norwalk Police Department are made from a list of candidates, ranked in order of scores which are derived from a combination of the score on a written test (65%), the score on an oral examination (15%), the average of the previous four performance evaluations (10%) and seniority (10%). At the time this court entered its consent decree, the various patts of the promotional process were weighed as follows: written test (60%), oral examination (10%), performance (20%), and seniority (10%). The new weighing procedures came into effect on July 1,1978 as the result of a collective bargaining agreement between the city and the police union. These changes have not previously been approved by this court.

2. The C.P.D. and the city followed the procedures detailed below in developing the sergeant’s examination:

a. The test-crafters drew from outside sources to develop a comprehensive task list for the position of sergeant. These sources included a list of 115 tasks compiled for the State on the State Police Sergeant project for which the C.P.D. had interviewed sergeants in five Connecticut municipalities; a list of 145 tasks already in the C.P.D. files; and task lists from the files of other localities (Toledo, Denver, Minnesota).

b. The tasks so listed were then categorized into eight conceptually related task areas.

c. The test-crafters then conducted a “brain-storming” session with Police sergeants to ascertain the relevant knowledges, skills and abilities (KSAs) for the Norwalk sergeants’ job. This process yielded 140 KSA statements, which were subsequently refined into 99 statements, and categorized into eleven KSA areas.

d. Next, the C.P.D. developed a questionnaire listing the 131 tasks in the eight task areas, and the 99 KSAs in the eleven KSA areas. The questionnaire was designed to measure the amount of time a sergeant in Nor-walk spends at the listed tasks (0-3) and the current supervisors’ impressions of the importance of good performance of these tasks to success on the job (1-5). With regard to the KSAs, the questionnaire was designed to measure current supervisors’ opinions as to which KSAs were ordinarily required for appointment to the job (1-5).

e. This questionnaire was distributed to all officers of the Norwalk Police Department at or above the rank of sergeant.

f. After allocating the KSA and task areas to each of the three different testing modes, Defendants’ Exhibit 3, Figure 2, the test preparers analyzed the 27 completed questionnaires to determine the proper allocation of items to each of the task areas scheduled for testing through the written examination. Defendants’ Exhibit 3, Figure 4.

g. In the final phase of test development, the C.P.D. gathered more than 900 individual questions from a variety of sources (C.P.D. item files, other state examinations and some new material) and selected a final pool of 222 items. These questions were reviewed by seven police experts from Connecticut, Defendants’ Exhibit 3, Figure 5, who rated each of the proposed questions in terms of its difficulty and its importance to effective job performance. The seven policemen and C.P.D. representatives also reviewed the keyed answers for possible ambiguities.

3. The examination as a whole is at an 11th grade reading level. The instructions are at a 10th grade reading level.

4. No test manual is available to test candidates, but a reading list has been compiled by one of the C.P.D. representatives, who is familiar with the content of the test.

5. The collective bargaining agreement in effect in Norwalk requires a minimum passing grade of 70% on all written tests. This fact was “taken into account” when the test was being developed, but no effort was made to show any statistical correlation between the difficulty of the test and 70% as the grade which would select qualified candidates.

6. At present, the Norwalk Police Department has 19 sergeants, one of whom is non-white.

7. A sample size of at least 30 is necessary to perform criterion-related validation.

CONCLUSIONS OF LAW

I. Burden of Proof

If a plaintiff can show that some promotional or hiring practice has an adverse impact (80% rule) on a minority group, see 29 C.F.R. § 1607.4D, the burden shifts to the defendant to show that the selection procedure bears a demonstrable relationship to successful performance of the job for which it is used. Albemarle Paper Co. v. Moody, 422 U.S. 405, 95 S.Ct. 2362, 45 L.Ed.2d 280 (1975); Griggs v. Duke Power Co., 401 U.S. 424, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971); Bridgeport Guardians v. Civil Service Commission, 482 F.2d 1333 (2d Cir. 1973). Though the Second Circuit has described the burden as “heavy”, Bridgeport Guardians, supra, 482 F.2d at 1337, it subsequently indicated that it meant nothing heavier than a burden of persuasion that the examination was substantially related to job performance. Vulcan Society of NYC Fire Dept. v. Civil Service Commission, 490 F.2d 387, 393 (2d Cir. 1973).

In this case, plaintiffs argue that the C.P.D. has previously prepared and administered promotional examinations for sergeant in Norwalk, which have yielded disparate results. Thus, they conclude, defendants bear a “particularly compelling” burden.

Defendants, in response, point out: (1) the promotion statistics are more reflective of discrimination in hiring than of discrimination in promotion and (2) the small sample size of the earlier tests makes it difficult, if not impossible, to show a disparate impact. Defendants claim, moreover, that any doubts should be resolved in their favor because they have voluntarily assumed the burden of validation according to federal guidelines.

In assessing these contentions as to the burden of proof, the court is guided by its finding that there has been no valid showing of disparate impact in previous promotional examinations prepared by the C.P.D., and thus will hold the defendant Police Department to a somewhat lesser burden on validation of this examination.

II. Oral Examination

Plaintiffs’ opposition to the oral examination is based primarily on the change in weight of the elements after the entry of the consent decree, and on the “free-wheeling”, subjective nature of such examinations, if they are conducted without reasonably definitive guidelines. Defendants have submitted a copy of a guidelines pamphlet which will be distributed to the oral examiners, and have pointed out that the panel of five, which will include two minority representatives, will receive instruction on question areas before the examination. In addition, they point out that the oral examination has not had a disparate impact in the past. The few black candidates who have reached the point of the oral examination have performed, on the average, better than the white candidates have performed, on the average. See 29 C.F.R. §§ 1607.4C, 1607.4D, 1607.6B(2). On the basis of that history, and because the plaintiffs have indicated that the parties have agreed to consult together on this portion of the promotion process, this court issues no ruling on the oral portion of the examination.

III. Performance Rating

Plaintiffs’ objection to the performance rating portion of the promotion process is limited to the statement that the defendants have not met the burden of demonstrating that the proposed testing procedures will not have a racially adverse impact. Defendants assert that the performance ratings of non-white candidates have in the past been slightly higher than those of the candidates as a whole, that each performance evaluation is signed by the ratee (and seen by the person at the time of the evaluation), and that the person who is evaluated can appeal an adverse evaluation through the grievance process.

This court finds that the defendants have met their burden as to the performance rating portion of the promotion process. See 29 C.F.R. § 1607.6B(2).

IV. The Written Test

A. Content vs. criterion-related validation.

The plaintiffs contend that past action of the defendants requires validation of the sergeant’s promotional examination through criterion-related validation procedures. However, defendants’ witness, Psychometrist Bruce Davey of the State Personnel Division, satisfied this court that criterion-related validation would be virtually impossible for a sample as small as the 19 Norwalk police sergeants — with only one non-white among them. The cases cited by plaintiffs as requiring the use of criterion-related validation are inapposite. Even assuming that criterion-related validation is the preferred method, it obviously cannot be required where it is not feasible. 29 C.F.R. § 1607.5A, 1607.14B(1). See Bridgeport Guardians v. Bridgeport Police Department, 431 F.Supp. 931 (D.Conn.1977). See also Vulcan Society of NYC v. Civil Service Commission, supra.

Defendants have made a convincing showing that criterion-related validation is not possible in the Norwalk Police Department; and they consequently have adopted a content validation procedure. In view of the acceptability of both methods, and of the technical unfeasibility of criterion validation, the court finds that content validation is proper. See 29 C.F.R. §§ 1607.5A, 1607.5B.

B. The sergeant’s examination is content valid.

To establish the content validity of an examination, the proponent must produce “data showing that the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated.” 29 C.F.R. §§ 1607.5B. The validity of the written examination should properly be more closely scrutinized than the oral examination and the performance rating, both because it is weighted the heaviest in the ranking process and because passing the examination is the threshold requirement for consideration for the sergeant’s job.¹ As a general proposition, the development of the Norwalk examination seems just as meticulous as the development of the test approved by Judge Newman in Bridgeport Guardians v. Bridgeport Police Department, supra, 431 F.Supp. at 936-37, and appears to meet all of the requirements of 29 C.F.R. § 1607.14C(2) and most of the requirements of 29 C.F.R. § 1607.14C(4).

The Norwalk test has principally been prepared by the C.P.D., a body with considerable experience, expertise and resources in the area of test development. They conducted extensive job analyses, drawing on the experience and resources of other municipalities as well as on the experience of Norwalk sergeants. The C.P.D. then devised an exhaustive questionnaire, which was completed by Norwalk sergeants and supervisory personnel. The answers to the questionnaire were analyzed by the C.P.D. to select the appropriate percentage of examination questions to be devoted to each subject area.

Plaintiffs attack the 100 multiple-choice question test designed by the C.P.D. on four grounds:

(1) They contend first, that the examination cannot be content valid because the questions have been adopted from prior non-validated examinations. The court finds this objection to be without merit. While validation obviously requires that individual questions be job related, the fact that they have appeared on non-validated examinations is simply irrelevant to the issue of whether the questions are job-related.

(2) They argue second, that defendants have not established a proper reading level for the examination and instructions. This objection has been met by evidence submitted by the defendant to show that the test is at an 11th grade reading level and that the instructions are at a 10th grade level. Plaintiffs have not to date contested that an 11th grade reading level is necessary for a sergeants’ job.

(3) Third, plaintiffs argue that defendants have not validated the ranking function of the examination as required by 29 C.F.R. §§ 1607.5G, 1607.14C(9). In essence, plaintiffs assert that defendants have not made the necessary showing that a higher score on the test will result in better job performance; nor have they proved that the examination will distinguish between qualified candidates according to valid criteria.

The federal guidelines provide that “[wjhere a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance.” 29 C.F.R. § 1607.14C(9).

The evidence in this case shows, at the least, that the percentage of items on the test relating to different task areas and KSAs was selected with reference to the impression of current Norwalk sergeants and superior officers as to the importance of certain KSAs and task areas to job performance. See Defendants’ Exhibit 3, pp. 4-5, Figures 2, 3, 4. Absent a criterion validation study, it would seem difficult to make a stronger showing of ranking validity. Of course, after this test is administered, post-test analysis of the job performance of successful candidates will support or undermine a conclusion of ranking validity. But the task here is, as accurately as possible, to determine prospective validity.

(4) Finally, and most importantly, plaintiffs object to the allocation of task areas and KSAs to the written and oral examinations. They point out that defendants have not produced any objective criteria applied to determine the testing modality for a particular KSA. Moreover, plaintiffs point out that as a result of this unsubstantiated allocation of task areas and KSAs to the different testing modes, the written test is barren of questions which test the Norwalkspecific KSAs of knowledge of police department procedures and knowledge of available resources. Plaintiffs conclude that, stripped of the components applicable to the job of sergeant in Norwalk, the proposed written examination is plainly not a representative sample of the position to be tested.

The justification offered by the C.P.D. for this decision is that departmental policy is best tested in an oral examination, because it is rapidly changing and otherwise may be outdated by examination time. Defendants assert that knowledge of available resources is specific to Norwalk, and therefore likewise best tested through oral examination. Defendants’ Exhibit 3, Figure 2.

Whether the omission of objective criteria in the apportionment decision vitiates the validity of the test requires a judgment call. There is merit in both positions. It might be more efficacious to test these Norwalk-specific KSAs through an oral examination, by asking the candidate how he/she would handle a specific situation. It is the type of issue as to which a multiple choice question would measure very little actual knowledge. On the other hand, the plaintiffs are properly concerned that the only two Norwalk-specific KSAs which the test group rated as fairly significant, will be tested in a less rigorous, less easily validated and more subjectively evaluated manner. This court will not substitute its less qualified judgment for that of the professionals herein involved.

C. The 70% cut off score is valid.

Plaintiffs’ final objection to the proposed promotion process is that defendants have not validated the minimum passing score of 70%. They acknowledge that the collective bargaining agreement requires a 70% score, but assert that defendants have made no showing that 70% will distinguish between qualified and unqualified candidates.

The federal guidelines provide that “[wjhere cut off scores are used, they should normally be set so as to be reasonable and consistent with normal expectation of acceptable proficiency within the work force.” 29 C.F.R. § 1607.5H. This requirement can be met either by choosing a relevant cut off point, or by fitting test difficulty to an arbitrarily-established cut off point. For example, Judge Newman upheld the 75% cut off required by Bridgeport Civil Service because the examination in question was constructed with the cut off point in mind.

Knowing that a passing score was required to be 75%, the McCann firm [test developers] selected questions on which the average correct-answer rate achieved by detective candidates throughout the country for all 100 questions was 68%.

Bridgeport Guardians v. Bridgeport Police Department, supra, 431 F.Supp, at 939. The arbitrary cut off in Bridgeport Guardians was validated in Judge Newman’s view by a showing that the relationship between the test and the cut off point was not arbitrary.

The validation showing in this case appears to be not as strong as the showing made in Bridgeport Guardians. Here, the only mention of the cut off in the validation proceeding was Mr. Davey’s testimony that the 70% passing score was “taken into account” when test questions were selected. In some instances, this testimony alone probably would not meet the requirement of showing that the relationship between the examination and the passing score is based on a sufficiently sound approach. Here, however, the court will validate the passing score of 70% by expressly applying less vigorous scrutiny to this passing score, because there has been no showing of a racially disparate impact. In validating and approving this sergeant examination of the Norwalk Police Department, the court refers the parties to its opening conclusion as to the lesser burden of proof which this court finds applicable to these defendants.

For the reasons stated above, the defendants’ Motion for Approval of Administration of Promotional Examination is GRANTED, with the proviso as to the oral examination as noted above.

It is SO ORDERED.

1 . The collective bargaining agreement requires that only those scoring 70 and above can be considered for a promotion.