We have consolidated your concerns into two separate issues that are addressed and discussed in detail in the enclosure to this letter.

1. Poor quality of reactor operator fundamentals training through an
erroneous reactor equation and
2. Identification of certain problematic generic fundamentals test items.

Background

Since 1989 the NRC has administered the Generic Fundamentals Examination (GFE) to facility candidates seeking reactor operator licenses. The NRC/GFE is a 100 point examination that measures candidate safety significant knowledge in three areas: reactor theory, components, and thermodynamics. Candidates must pass the GFE with a score of 80 percent before they are eligible to take the balance of the NRC site-specific license examination administered at their facility.

The NRC/GFE is developed through contractor support but is carefully reviewed and approved by the NRC before it is administered. Under an understanding with the Institute for Nuclear Power Operations (INPO) in 1991, the NRC has agreed to use the INPO Generic Fundamentals Catalog -- a separate catalog distributed to its member facilities -as source material for developing the NRC/GFE. To this extent, our contractor draws approximately 40 test items from the INPO catalog, modifies them through changes to the question stem and distractor options to meet NRC standards of psychometric quality, then validates each item for technical accuracy and generic applicability, and finally submits the entire examination to the NRC for approval. Each NRC/GFE test item receives multiple levels of review and approval before it is accepted and finalized within the examination. After items have been administered on the NRC examination, they are fed back to INPO for inclusion into-their catalog updates.

Although, as stated above, a detailed discussion of both issues appear in the enclosure, a summary of each issue is included here.

Summary

Issue 1: Erroneous equation

In your January 10th and February 28th letters you included the following equation, drawn from the INPO Catalog, which you stated is incorrect.

We do not agree with your view entirely. The Generic Fundamentals Examination Equation Sheet that accompanies each NRC/GFE contains two equations (the second, an inverted form of the above) which relate stable reactor period, Tau, to reactivity, rho, viz.:

	(1)
	(2)

In our analysis, we found the equations to be accurate and appropriate. We agree that when they are used in questions, they should be couched in the context of a commercial power plant application and should contain the correct caveats (e.g., "following a step insertion of reactivity" vice a continuous withdrawal) so that the equations are applicable.

Issue 2: Problematic test items

In your January 31st letter you identified 11 test items from the INPO Catalog, 89-004, as problematic and in your subsequent February 28th letter you provided additional test items with references from the Academy Document 89-004 that you believed were also problematic, having had either implausible question stems or incorrect answers; those items were 10, 20, 21, and 27-31. In addition, you included in your letter a copy of an October 19, 1991 letter which you had sent to INPO in which you further identified by reference three additional problematic items: 2, 11, and 24. In the enclosure of this letter, each item is identified, accompanied by your comment and proposed resolution, and is followed by the NRC discussion and resolution.

In our review we found no instances of any of the problem items in the NRC/GFE examination bank. Moreover, in every case, we agreed that the item had problems and would not appear on any NRC examinations; however, we did not always concur with your reasoning or basis for our decision. About half of the issues you raised in the comments had merit; however, several of the questions were found to be inadequate for other reasons -- usually due to poor wording in the stem or distractor options.

We also reviewed the most recent INPO Generic Fundamentals Catalogs (BWR and PWR) updates to assess the presence of those items. Only one of the challenged questions still exists in the September 1994 INPO BWR catalog. This indicates that most of the concerns were probably addressed at an earlier date, perhaps in response to the same concerns you raised several years ago. However, in several cases there were a number of related questions located in the catalogs that contain the same problems.

Since the NRC routinely uses the INPO examination bank as source material for NRC examination development, we have flagged those items and will recommend to INPO that they delete or modify the items in their bank accordingly.

In summary, we appreciate your bringing to our attention problematic test items. Although the items you submitted do not appear in our bank, we have taken steps to eliminate or correct a number of generic psychometric problems that those items had and thereby reduce the possibility that those items could potentially have appeared on future NRC examinations. Although we have not agreed with every basis of your concern, your attention has caused us to re-evaluate the generic issues you raised; consequently, we will continue to review items in our bank.

I hope this information satisfactorily resolves your concerns.

Enclosure: As stated (pp 1 -10)

.