M05: Experimental Design Activity to Compare Performance of Generative AI Platforms


By Monnie McGee, Bivin Sadler


Information

A college education is one reliable way of climbing the social ladder. US News & World Report updated its rankings in 2023 to include social mobility, and many institutions of higher education are paying more attention to recruitment of first­ generation college students and talented students from disadvantaged backgrounds. The inclusion of these students requires that instructors consider access issues that were not previously considered. For example, students from advantaged backgrounds will be able to pay for access to ChatGPT 4.0, which $20 per month for a subscription, and is touted to be better than ChatGPT 3.5, which is free. Nine students in a required graduate course in classical statistical methods and experimental design were given a project to explore whether and how much the choice of a generative Al platform matters in the context of content in a typical statistics curriculum. The students are first-year students in the PhD programs in statistics and biostatistics at SMU, a four-year comprehensive research university. The content compared was up to the student, as well as the design, as long as at least two generative Al programs were compared. Designs and results from select student projects will be presented and discussed.


Recording