Writing Evaluations

To develop an assessment of department or team writing competence, the Center will evaluate individual employees' writing and conduct written and oral interviews with employees to determine the effectiveness of written communication within the company and any problems that might be arising, including miscommunication, poorly written documents, delays in communication, inadequate explanations and responses, and overabundance of correspondence, especially e-mail.

The Center will provide a report for individuals and the department or team with suggestions for changes that will improve communication.

Evaluating the effectiveness of writing training

We will evaluate the effectiveness of writing training whether we provide the training or not.

To evaluate the effectiveness of training, the company may provide pre-training and post-training writing samples with codes to identify each sample, but without writers' names or information that might indicate which are pre-training and which are post-training. The Center will evaluate the samples and return them with ratings. After the company has the evaluations in hand to ensure objectivity, the company or the Center can use the codes on the returned samples to sort out pre-training and post-training samples and evaluate the results of the training by comparing individuals' progress and averages for the entire group.

Center professional staff are experienced in performing such studies using raters and have published articles about the methods in professional media. Dr. Hogan's Ph.D. dissertation was in the rating of writing and analysis of results. Center professional staff will walk the company through the process and set it up. If we do the analysis after ratings, we will provide a report of the results.

If requested, the Center will contract to develop a pre-post testing design with two or more raters, rater training, inter-rater reliability coefficients, intra-rater reliability coefficients, and a report on testing methodologies for inclusion in the company or institution's study report.

Methods of Diagnosing Writing Competence

A qualified writing specialist evaluates each writer's competence using a writing sample. It may be normal business writing the person is creating or a special sample written from a case study for this evaluation. The result may be any of the following:

A formal report.
List of skill areas the person needs to work on with corrections and explanations in the writing sample.
Ratings for each of the eight critical dimensions required for clear, correct writing and an overall rating for the writing sample.
A simple overall rating for the writing for use in evaluating a large sample of pre- and post-test writing samples.

Option 1: A formal report about the person's writing.

The formal report contains the following:

A summary of strengths and weaknesses.
A detailed listing of the specific problems and areas of strength in the eight critical areas: clarity, content, format, organization, planning, structure, style, and usage.
A copy of the document with interlinear notations about the writing using Word's review function.
Recommendations for training.

Option 2: List of skill areas the person needs to work on and notations about the writing in the sample.

A list of the skill areas the person needs to work on with explanations and examples of each area. The evaluation includes a copy of the writing sample with notations in the sample about the skill areas the person needs to work on and corrections showing how the writing should be changed.

Option 3: Simple ratings without a formal report.

The writer or administrator receives ratings in each of the eight areas critical to clarity and correctness in writing (clarity, content, format, organization, planning, structure, style, and usage), an overall rating for the sample, and a copy of the document on which the rater has written notes to assist in the rating. The notes are not meant to be a formal report. The evaluator suggests training that would benefit the writer.

Option 4: A rating for the writing sample without a formal report.

The writer or administrator receives a single rating for the sample for use in evaluating groups of trainees.

Meanings of the Ratings

Rating of 1: The writer has serious language and usage problems or inability to structure and organize writing to communicate clearly. This level of writing is characteristic of nonnative speakers with inadequate training in English as a second language, people with an impoverished educational background, and people with limited verbal aptitude. This person can be helped only by extensive one-on-one training and diligent work. The person may not rise above Level 1 in writing competence and likely will not rise above Level 2.
Rating of 2: The writer has a number of different language and usage problems or does not structure and organize writing to communicate clearly. Supervisors have observed problems in his or her writing or the person has received unfavorable comments about his or her writing from colleagues or customers. This person can be trained to overcome most of the writing problems and rise to a Level 3 with a 70 percent to 90 percent level of competence. This writer probably will not rise to a Level 4 unless the person and company are willing to involve the writer in an extended period of training.
Rating of 3: The writer's documents are clear and correct most of the time, but he or she displays a small number of language and usage problems, especially in the areas in which business writers normally experience difficulty such as commas and other punctuation. This person can learn the skills to become a Level 4 writer. With additional training in usage and editing skills, the person could then advance to a Level 5.
Rating of 4: The writer is very competent in all areas of writing and does not need training in basic writing skills or business writing skills. He or she probably has a native verbal ability. This person will benefit from the advanced courses, such as Editing Skills and Proofreading skills. With advanced training, this person could advance to Level 5.
Rating of 5: The writer is unusually competent and has no flaws in his or her writing. He or she communicates in creative ways to enhance the reader's understanding and appreciation of the content. This writer is able to edit and proofread others' work. The person may train others in writing skills.

Fees

Writing evaluation with a formal report (two to five pages)	$75
List of skill areas the person needs to work on and notations about the writing in the sample (one to two pages).	$50 ($45 for 20 or more samples)
Rating for each of the eight critical areas of writing competence (clarity, content, format, organization, planning, structure, style, and usage) without a report but including the rater's notations in a copy of the writing sample (one to three pages)	$39 ($35 for 20 or more samples)
Simple overall rating for writing samples in groups of 20 or more samples (one or two pages)	$23 per sample
Developing an evaluation design for a company or institution, training raters, providing inter-rater and intra-rater reliability coefficients, and performing blind ratings:	On contract, with fees depending on the contract

Signing Up for Evaluations

E-mail the Center at center@writingtrainers.com