Rejection: LM 4
Ryan and Tippins article was interesting and clearly alluded to the limitation of references and cognitive/personality skills tests in assessment when not performed properly. Further these readings also indicate that the battery of questions for assessing personality need to be implemented fully, rather than making an interpretation based on a few isolated questions.
With that perspective I would be disappointed if the rejection was based upon an assessment that was made on cognitive or personality testing when a full complement of questions or a comprehensive test was not administered. This occurs not too infrequently in work place, for the convenience of saving time. In addition, most interviewers, especially in medicine are experts in their medical field; they have little back ground in cognitive and personality testing. While some of these tests have good validity in identifying cognitive and behavior attributes, it requires an expert in this field to be able to properly implement and interpret the data from such tests.
I would be not pleased if the rejection was also due to referee’s recommendation or lack thereof. As part of my interviews for my current job (Chair), the Dean of the medical school called few selected other nationally known leaders in the specialty. It was a big concern for me, as the referee’s he chose may not have known me well or well enough. There expertise may have been different than mine and our paths may not have crossed beyond being acquaintances at national meetings. Or, possibly, they may have been my competitors. I would have found this process to be unfair as the interviewer (dean) is frequently looking to seek out any negatives in the candidate---anything short of a glowing enthusiastic endorsement from the person on the other side of the phone is generally considered a negative for the chair position. Unless the referee is unbiased and understands the implications of his statements, this brief conference call to a few selected opinion leaders has potential to be very detrimental to the selection process. A possible way to mitigate this would be to cast a wider net of randomly chosen referees, and/or to include in the list some of the referees suggested by the candidate. Regardless, the reliance on referees as the sole or primary criteria for determining national pre-eminence in your field is fraught with potential errors.
Additional concerns I have with selection process are interpretations that come out of unstructured interviews. As indicated by the Ryan and Tippins article, unstructured interviews have relatively low validity. When criteria for evaluation are not established a priori, it is very difficult to measure the outcome of a response. Further, depending on the breath and scope of the questions that are posed in an unstructured interview, the actual merits of the candidate (me) may not have been fully realized. That would have certainly frustrated me. Fortunately, several major medical schools have realized perils of unstructured interviews and now have expert consultant groups guide selection of candidates for higher executive positions.