Facts In Action

Making it Count is a series of articles designed to help you develop ways to measure outcomes in your program or family child care home. If you would like to receive earlier issues of Making it Count, please contact Erika Argersinger at (617) 695-0700 x271, or by email at eargersinger@associatedearlycareandeducation.org. You can also find earlier issues on-line at factsinaction.org/mcount/making-it-count.htm.

Previous issues of Facts in Action have discussed the preliminary steps for developing an outcome measurement system for your program: assembling a working group to develop a measurement system, choosing which outcomes to measure, and choosing indicators for your outcomes. The next step in developing a measurement system is deciding where you will obtain the data for each indicator, and how you will collect and record these data.

The first task for the working group when preparing to collect data is to identify data sources within your program. Most child care centers or homes collect some information on the children in their programs - general administrative information, specific observations of children's activities or interactions, or formal child assessments. When evaluating a potential source of data, it is important to consider whether the data source will provide useful, reliable information related to the outcomes you have identified. That is, the working group should consider whether the data measure achievement of an outcome and whether the information is useful to teachers and directors for program improvement.

Once the group has identified data sources, it may need to develop a method for collecting and recording this information - a data collection instrument. Developing a good instrument can be very time-consuming. Rather than creating an instrument from scratch, it may be helpful to find an existing instrument and modify it for your program's specific needs. Good resources for finding existing data collection instruments include: other programs, professional organizations, accrediting bodies, and the internet (see sidebar for specific resources). In addition, many child care programs use some sort of child assessment tool, whether it be a checklist or a rating chart. If your assessment tool works well for your program and captures the information you are interested in collecting related to outcomes, it can be used as the basis for a data collection instrument.

To check that an instrument is collecting the information needed to measure program achievement, the working group should ask itself:

• Does the instrument gather the information necessary to determine if outcomes are being reached - that is, does it measure the indicators we've already identified?

• Can we identify specifically which question in which instrument will provide data on each indicator?

Once you have chosen a data collection instrument that fulfills the needs of your measurement system, the group has to develop a step-by-step plan for collecting data, training the individuals who will be collecting the data (e.g. teachers), and providing technical assistance. These tasks will be discussed in the next issue of Making it Count.

Action Steps

Get a copy of the United Way of America's handbook, Measuring Program Outcomes: A Practical Approach. To order a copy, call (800) 772-0008 and request item number 0989. Get a copy of the United Way of Massachusetts Bay's handbook, Outcome Measurement in Child Care Programs: A Workbook for Practitioners. To order a copy, call (617) 624-8000. Start thinking about the different sources of data in your programs. Ask yourselves if the data source provides useful, reliable information related to your desired outcomes. Start gathering data collection tools in order to compare and evaluate their strengths and weaknesses. You might want to adopt one of these tools in whole, or modify one to suit your measurement system's specific needs.

Sources:
Measuring Program Outcomes: A Practical Approach, United Way of America, 1996; Outcome Measurement in Child Care Programs: A Workbook for Practitioners, United Way of Massachusetts Bay, 2000.

Facts in Action, February 2001