Catherine Hu considers the effects of platform design and artificial intelligence on the advertising industry

When you shop online, do you notice the product image first—or the description? According to new research from assistant professor Catherine Hu, Ph.D., that split-second difference can significantly influence how consumers understand and respond to a product. In her publication, “Order Matters: The Marginal Value of Visual and Textual Information in Sequential Product Information Presentation,” Hu explores how the sequencing of visual and textual content shapes user engagement with online advertisements.

“That research paper primarily investigates how changing the sequence for presenting different formats of product-related information – specifically visual and textual information – can influence our responses or reactions to products,” Hu said. “In one condition, people saw the textual description first followed by the images. In the second, they saw the images first followed by the text.” 

An online survey analyzed this relationship between digital format and consumer engagement, Hu said. Participants viewed a series of advertisements featuring various visual and textual layouts. They then answered a set of related questions that offered insight into their perceptions, attitudes and behavioral intentions surrounding each of the products.

“Our major finding is that sequence does influence how people respond to product information,” Hu said. “When we present the textual information first, people find it easier to understand, easier to learn about and easier to get information on the product. It also influences their attitude and purchase intention.”

For Hu, these observations hold particular relevance today. Amid the proliferation of internet retail trends, virtual platform and application design is becoming increasingly critical to marketing success, she said.

“This research study highlights the importance of considering the layout, the format and the structure of how we want to design an online interface for today’s consumers,” Hu said. “Ultimately, the format of how information is presented is as important as the actual content of online information.”

Beyond this project, Hu’s broader body of research emphasizes topics of user control in the internet era. She primarily seeks to understand how AI-powered systems may alter or even undermine a consumer’s autonomy over their own digital experiences.

Hu said that as media platforms continue to evolve intellectually, AI tools take matters of agency and decision-making from the hands of users.

“Let’s say I am using an AI assistant to help me do something,” Hu said. “Sometimes, it can feel like I am not really doing that thing, but rather that someone else is giving me orders and instructions to work on a project. So, we start to see a shift in control and autonomy.”

In Hu’s pursuit of these dynamic themes, a number of moments stood out to her as noteworthy, including the integration of eye-tracking technology into her academic endeavors.

According to Hu, eye-tracking tools — devices that monitor users’ eye movement across content on a screen — remain one of the most compelling and valuable methods for data collection in the field.

“Eye tracking is a relatively more objective way to measure our attention and how we actually process information,” she said. “If you think about attention, it is very intangible, it is very hard to capture. By using that technique, we can see how people’s eyes move when they are working with digital media tasks. So, I think it is a pretty interesting way to give us some biometric insights on how people interact with media content.”

Now, with an eye toward the future, Hu eagerly awaits the evolution of advertising and media industries alike. AI tools are redefining the observable relationship between digital audiences and marketing professionals, she said.

“Personalized advertisements tailored to your chatting history with ChatGPT or other AI tools are coming,” Hu said. “I think that will be something interesting to study, as well as how people respond to that.”

Throughout this evolutionary period, though, young scholars must remember to protect and enhance their own independence and informed judgment.

“AI should be a tool for you, but keep your own autonomy,” Hu said. “I use AI on a daily basis for all kinds of tasks, but I feel that we should maintain our own critical thinking in using it. Don’t let it replace your own independent thinking.”