9 Training and Professional Development
Chapter 10 of the Dynamic Learning Maps® (DLM®) Alternate Assessment System 2015–2016 Technical Manual—Science (Dynamic Learning Maps Consortium, 2017) describes the training that was offered in 2015–2016 for state and local education agency staff, the required test administrator training, and the optional professional development provided. This chapter presents the participation rates and evaluation results from the 2021–2022 use of the optional instructional professional development. This chapter also describes the updates made to the professional development system during 2021–2022.
For a complete description of training and professional development for DLM assessments, including a description of training for state and local education agency staff, along with descriptions of facilitated and self-directed training, see Chapter 10 of the 2015–2016 Technical Manual—Science (Dynamic Learning Maps Consortium, 2017).
9.1 Updates to Required Test Adminsitrator Training
Two main revisions were made to the DLM Required Test Administrator Training for 2021–2022. Slide content and narrations were revised to complement each of these revisions.
First, all modules were revised to comply with the Americans with Disabilities Act (ADA) guidelines. These revisions include alternative text for all images and tables, a unique title for each slide, and more succinct slide and narration content.
Second, all screenshots of the DLM website were removed from the modules in anticipation of an overhauled website launch in late July of 2021. The screenshots were no longer necessary, as all resources on the revised site are easier to find using filters or an enhanced search bar. The revised modules focus more on the name and usefulness of each resource rather than relying on directions for how to navigate the website.
Additionally, one minor revision was made to the slide and narration regarding the Security Agreement. The Security Agreement for 2021–2022 no longer has a choice to reject the agreement, only to a box to accept the agreement. Therefore, the slide and narration were revised to reflect this change.
The total time needed to complete the training, activities, and post-tests is still under three hours.
9.2 Instructional Professional Development
The DLM professional development system includes eight modules that address instruction in science and support educators in creating Individual Education Programs that are aligned with the DLM Essential Elements. While the modules were originally intended for educators who administer DLM assessments, demographic information suggests that preservice educators, related service providers, parents, and others also accessed and completed the modules.
The professional development system is built in WordPress, an open-source website content management system. The professional development modules and instructional support materials are available for anyone’s use at https://dlmpd.com or through a direct link from the DLM website. These DLM professional development modules address instruction in science. The modules also address processes educators can apply to create Individual Education Programs that are aligned with the DLM Essential Elements and supports they can provide to address the communication needs of the students they teach. Finally, the modules help them understand the components of the DLM assessment system more completely.
To support state and local education agencies in providing continuing education credits to educators who complete the modules, each module also includes a time-ordered agenda, learning objectives, and biographical information regarding the faculty who developed the training modules. There are a total of eight modules, which are described in section 9.2.1.2 of this chapter.
The eight modules are available in both self-directed and facilitated formats. The self-directed modules are available online, on-demand. The interactive modules include a combination of video-based content, embedded activities, and, for participants who would like to receive a certificate documenting their successful completion of the module, a five-item pre- and post-test. These certificates are sent directly to each participant’s email when they score 80% or higher on the post-test.
Modules in the facilitated format were created for groups and by individuals who prefer to learn by reading the contents rather than interacting with video and other package contents.
In addition to the eight modules, the instructional professional development site provides instructional resources for educators. These include DLM Essential Element unpacking documents; vignettes that illustrate shared reading with students with the most complex needs across the grade levels; supports for augmentative and alternative communication for students who do not have a comprehensive, symbolic communication system; alternate pencils for educators to download and use with students who cannot use a standard pen, pencil, or computer keyboard; and links to Pinterest boards and other online supports.
During the 2021–2022 school year, teams at ATLAS worked in cooperation with the professional development team at the University of North Carolina (UNC) at Chapel Hill to develop a new science professional development module titled “Science and Engineering Practices #2: Developing and Using Models.” This module focuses on teaching the “developing and using models” component of one science and engineering practice through the use of a writing framework. The module also includes guidance for teaching the science and engineering and framework through differing levels of complexity using DLM Science Essential Elements and linkage levels. The writing and production process of the module included collaboration between teams to ensure alignment with existing English language arts (ELA) and mathematics learning modules for a more streamlined DLM learner experience. The module will be released for the 2022–2023 year, and evaluation data will be included in future technical manual updates.
9.2.1 Professional Development Participation and Evaluation
There are two ways in which test administrators and educators may complete professional development modules: required test administrator training or optional professional development. As described in Chapter 10 of the 2015–2016 Technical Manual—Science (Dynamic Learning Maps Consortium, 2017), some states require professional development modules to be completed as part of the required test administrator training. States can require certain modules be completed by new test administrators, returning test administrators, or both. Test administrators completing professional development modules as part of the required test administrator training access the modules through the Moodle training site where the rest of the required test administration training is located. The second way in which professional development modules are completed is through the DLM professional development website. The modules on the professional development website may be accessed by anyone and can be completed at any time. Additionally, participants completing modules on the professional development website are administered a short evaluation survey following the module. With the exception of the evaluation survey, the content of the modules is identical between the required test administrator training and the professional development website.
9.2.1.1 Required Professional Development Participation
A total of nine states required at least one professional development module as part of their required test administrator training. The modules included in the required test administrator training are required of all relevant test administrators (i.e., new or returning, as specified by the state). For example, a test administrator who only administers science assessments may still be required to complete a module on instruction for ELA. Table 9.1 shows the number of modules required, by state, for new and returning test administrators, as well as the total number of modules completed. In total, 21,888 professional development modules were completed by 4,431 new and 3,033 returning test administrators as part of the required training.
|
New test administrators
|
Returning test administrators
|
||||
|---|---|---|---|---|---|
| State | Required modules | Test Administrators | Required modules | Test Administrators | Total modules completed |
| Alaska | 1 | 84 | 1 | 130 | 214 |
| Delaware | 2 | 359 | — | — | 718 |
| Maryland | 4 | 1,466 | — | — | 5,864 |
| New Hampshire | 1 | 152 | — | — | 152 |
| New Mexico | 3 | 824 | — | — | 2,472 |
| Oklahoma | 5 | 649 | 5 | 1,305 | 9,770 |
| Rhode Island | 1 | 110 | 1 | 190 | 300 |
| West Virginia | 2 | 203 | 1 | 364 | 770 |
| Wisconsin | 1 | 584 | 1 | 1,044 | 1,628 |
Table 9.2 shows which modules were required for new and returning test administrators across all states choosing to include professional development modules in the required training. For example, the DLM Essential Elements module was required for new test administrators in four states and was required for returning test administrators in two states.
| Module | States requiring for new test administrators | States requiring for returning test administrators | Total modules completed |
|---|---|---|---|
| DLM Claims and Conceptual Areas | 1 | — | 1,466 |
| DLM Essential Elements | 4 | 2 | 3,993 |
| Effective Instruction in Mathematics | 3 | 2 | 3,987 |
| Individual Education Programs Linked to the DLM Essential Elements | 4 | 2 | 3,437 |
| Principles of Instruction in English Language Arts | 3 | 2 | 5,048 |
| Universal Design for Learning | 1 | — | 824 |
| Who are Students With Significant Cognitive Disabilities? | 4 | 1 | 3,133 |
9.2.1.2 Optional Professional Development Participation
In addition to the modules included in the required test administrator training, the DLM professional development website contains modules specific to science. Table 9.3 shows the number of individuals who completed optional professional development science, as well as the total number of test administrators from each state who had a student assigned to them for the DLM assessment. In total, 450 science modules were completed in the self-directed format from August 1, 2021, to July 31, 2022. Since the first module was launched in the fall of 2017, a total of 3,502 modules have been completed on the professional development website.
| State | Participants | DLM test administrators | Total modules completed |
|---|---|---|---|
| Alaska | 0 | 123 | 0 |
| Arkansas | 0 | 759 | 0 |
| Delaware | 0 | 167 | 0 |
| District of Columbia | 5 | 93 | 17 |
| Illinois | 1 | 2,005 | 1 |
| Iowa | 1 | 543 | 1 |
| Kansas | 1 | 544 | 1 |
| Maryland | 4 | 1,046 | 16 |
| Missouri | 3 | 1,114 | 5 |
| New Hampshire | 0 | 193 | 0 |
| New Jersey | 3 | 1,859 | 4 |
| New Mexico | 1 | 463 | 7 |
| New York | 4 | 3,978 | 14 |
| North Dakota | 0 | 165 | 0 |
| Oklahoma | 3 | 989 | 10 |
| Pennsylvania | 4 | 3,253 | 25 |
| Rhode Island | 1 | 174 | 1 |
| Utah | 7 | 714 | 14 |
| West Virginia | 1 | 342 | 3 |
| Wisconsin | 38 | 1,026 | 117 |
| Non-DLM state and other locations | 55 | — | 214 |
| Note. Participant counts may include individuals who are not educators or test administrators (e.g., pre-service educators). |
To evaluate educator perceptions of the utility and applicability of the modules, DLM staff asked educators to respond to a series of evaluation questions on completion of each self-directed module. Three questions asked about importance of content, whether new concepts were presented, and the utility of the module. Educators responded using a 4-point scale ranging from strongly disagree to strongly agree. A fourth question asked whether educators planned to use what they learned, with the same response options. During the 2021–2022 year, educators completed the evaluation questions 69% of the time. The responses were generally positive, as illustrated in Table 9.4. Across all science modules, 62% of respondents either agreed or strongly agreed with each statement.
To evaluate the consistency in the ratings for each module, we calculated Cronbach’s (1951) alpha from the four items for each module using all ratings from fall 2017 through the 2021–2022 year. Across all modules, alpha ranged from .92 to .97 with an average value of .96, suggesting high internal consistency in responses.
9.3 Conclusion
During 2021–2022, the required test administrator training had minor revisions to increase accessibility and focus on the most relevant content for those administering the DLM assessments. Instructional professional development modules continued to be available, and educators provided consistently positive feedback regarding the importance and relevance of the modules. Finally, a new module was developed, which will be available to eduators in future years.