Today’s guest post is brought to you by Heicke Hoffman, Professor of Statistics. Heicke is so new to UNL that only her salary info has been counted for STATS, not her research productivity. You might be able to guess that the UNL STATS department is really good at statistics. But what you might not know is that Heicke and her colleagues are also incredibly generous with their knowledge and time. It’s almost like they live to help people with their statistics problems. Heicke helps out today with unpacking problemsin the Academic Analytics data for TMFD and offers an important CORRECTION to an early post I made based on faulty and incomplete data and explains why you should always check your work.
What is TMFD?
The Chancellor’s proposal for the UNL Budget Cut proposes the elimination of the Department of Textiles, Merchandising, and Fashion Design (TMFD). Again, as faculty member Michael Burton reminded everyone at the Board of Regents’ meeting last week.
TMFD serves an important role in Nebraska, as the Department Chair, Jennifer Johnson Jorgensen explains:
In Nebraska, adopting sustainable practices in textile production, such as using agricultural waste and recycled materials to create new fibers, employing zero-waste apparel design, using effective technologies (such as CLO 3D and merchandising analytics), and employing effective textile conservation techniques, supports vibrant and sustainable communities. This shift also creates new entrepreneurial and financial opportunities, creating jobs that are both environmentally and economically beneficial, thereby positioning Nebraska as the leader in the growing global market for recycled, sustainable textiles and retail businesses.
Like 5 out of the 6 departments identified for elimination, it is among the 10 smallest by number of faculty at UNL. But TMFD is a small department doing big things.
These big things have not been well captured in the metrics used to justify these department cuts. In further examining the peer group from the recent post, You Can’t Do that with Statistics, TMFD is in the bottom quintile – ranked 25 out of 41 peers. … except, there should be 127 peer departments! What explains that descrepancy?
Compared to Who?
Why does Academic Analytics only have 41 peers for the TMFD Department? Looking more closely at Academic Analytics data, it appears that the peer group for TMFD is identified as “Consumer and Human Sciences, various.”(A word of caution: whenever someone describes your work as ‘something with’, ‘various’, or ‘whatever’, they clearly don’t know what you are doing.) But I just told you, the department is Textiles, Merchandising, and Fashion Design. None of those words are the same as Consumer and Human Sciences.
So how come Academic Analytics doesn’t know what TMFD does?
CIP CODE Confusion
Academic Analytics uses CIP (Classification of Instructional Programs) codes defined by the National Center for Education Statistics to create peer groups across higher education institutions. Glancing over the Textile-related CIP codes, it is clear that there are more fitting choices than ‘Consumer and Human Sciences, various.’ Here’s a selection:
| CIP Code | Program Name|
| 19.0901 | Apparel and Textiles, General|
| 19.0902 | Apparel and Textile Manufacture|
| 19.0904 | Textile Science|
| 19.0905 | Apparel and Textile Marketing Management|
| 19.0906 | Fashion and Fabric Consultant|
19.0999 | Apparel and Textiles, Other
It seems like there are plenty of more closely related CIP codes for TMFD. So why is TMFD misclassified?
Well, it turns out, the institutions submit the CIP codes of their programs to Academic Analytics. And whoever was in charge of doing that didn’t do it accurately and no one was checking their work. Oopsies!
Weighing Scholarship Across Fields
Why is it important to have the correct CIP code that provides the correct comparison group? Let’s go into a bit more of how scholarship is evaluated in Academic Analystics.
The Scholarly Research Index (SRI) is a measure developed by Academic Analytics to evaluate the research performance of individuals and entities with respect to
- scholarly products, such as conference proceedings, research articles, books, and book chapters,
- recognition from the community in form of citations and awards, and
- federal sponsoring of research projects measured by the number of grants and their amounts.
Different disciplines have different standards as to what constitutes excellence. And different disciplines have access to different amounts of resources (see NCES for the amount of federal funding by discipline in 2023). Academic Analytics deals with that by using discipline-adjusted weights for each of their measures of scholarship, including research and creative products.
These weights are so fine-tuned that some disciplines even have multiple sets of weights. For example, Sociology has two recognized sub-fields: one that focuses on research papers and the other emphasizes book writing. Looking at the table below, this is clear in the weights. It is also clear in the weights used for TMFD’s Scholarly Research Index are fairly generic, spead across all categories. These weights traslates roughly to “just be excellent at everything all at the same time,” a high bar in any discipline.
Using generic weights and the wrong comparison group misrepresents TMFD in the Academic Analytics data fed into the administration’s metrics.
Staying in the Correct Weight Class
Performance among peers determines the national ranking of departments. Being evaluated against your peers is the only fair comparison. It’s like wrestling within your weight class; anything else would be unreasonable.
So we know from the incorrect number of comparison units and the incorrect CIP code that TMFD is classified incorrectly. Without the ability to correct the data in Academic Analytics, the result remains: unfair, unreasonable, unjust, unwarrented. And just plain wrong.
