The Human Experience: Are we Losing the User-First Mindset as AI Systems Assume Greater Roles in our Daily Lives?
Kanei-ji Temple, Taito City, Tokyo, Japan (Credits: Ana-Paula Correia)
During a recent trip to Tokyo, Japan, I was struck not only by the impressive technological infrastructure and the seamless public services but also by how everything, from the toilets to the subway system, seemed designed with profound respect for the human experience. These systems were not merely efficient; they were imbued with empathy. It led me to reflect on how far we’ve come and perhaps how far we’ve strayed from the user-first mindset that originally guided much of technological innovation. As artificial intelligence (AI) begins to permeate every corner of our lives, particularly in education, the question becomes increasingly pressing: Are we losing sight of the human?
Tokyo and the Comfort of Thoughtful Design
For ten days, I lived in a space where design anticipated my needs before I could articulate them. The electric kettle in the hotel room had built-in safety features to prevent burns. Power outlets were abundant and intuitively placed. The housekeeping system operated unobtrusively, respecting the rhythms of my day. Outside, cars glided silently along the streets, yielding politely to pedestrians. Drivers anticipated movement with such care that honking felt like a foreign concept.
None of these features are groundbreaking on their own. But together, they created a fabric of everyday life that felt profoundly considerate. This is not the sort of “innovation” that makes headlines, but it is the kind that changes lives quietly and persistently.
A Brief History of the Human Experience in Design
The concept of “user experience” has evolved significantly over time. Its roots can be traced back to World War II, when engineers began to apply what would now be called human factors engineering to reduce pilot error in aircraft cockpits. In the 1980s and 90s, the rise of personal computing brought with it a new urgency to design systems that ordinary people, not just trained professionals, could use efficiently. Usability became a key metric, and eventually, user experience (UX) evolved to encompass not only functionality but also emotion, satisfaction, and the intangible elements of interaction.
As Don Norman (2004) emphasizes, the importance of designing products that not only function well but also bring joy and beauty to our lives lies at the heart of emotional design, where aesthetics and user experience are as critical as usability and performance.
In Tokyo, I was living in a city that had internalized this philosophy and embedded it into the very texture of its public and private spaces.
What Is the Human Experience?
The human experience is more than convenience or usability. It is the quiet joy that emerges when objects and systems “just work.” It’s the sense of being cared for by design. The feel of well-made fabric against your skin, the predictability of traffic flow, the trust that your ATM card won’t be devoured by a buggy interface. These are not small things. They are, in fact, the architecture of our daily emotional lives.
Travel—especially across cultures—amplifies our awareness of this. We begin to see how some systems accommodate us effortlessly while others put friction in our path. We become aware of the care, or lack thereof, embedded in everyday objects and experiences.
The Rise of Artificial Intelligence Since 2020
Since 2020, artificial intelligence has moved rapidly from the realm of research and experimentation into widespread public adoption. Generative Pre-trained Transformer 3 (GPT-3) was released in June 2020, marking a significant leap in large language model capabilities. Shortly afterward, tools like ChatGPT (launched in November 2022) and Midjourney (released in July 2022) began reshaping content creation, research, coding, and education. Artificial intelligence moved from backstage to center stage, becoming an integral part of our lives as teachers, researchers, content creators, coders, journalists, and more.
AI can scale personalized learning, provide real-time feedback, automate administrative tasks, and offer tailored content. Yet, these advantages come with an urgent challenge: ensuring that AI systems do not optimize for efficiency at the cost of the human experience.
Are We Losing the User-First Mindset?
As AI systems take on larger roles, particularly in educational technologies (EdTech), there is a growing concern that the user-first mindset may be slipping away. AI often operates behind the scenes, making decisions on behalf of users, mostly without transparency or context. When a student receives a grade from an algorithm or a recommendation for a learning module, is there a sense of care? Of understanding? Or is it simply an output that feels cold, even if accurate?
User experience in EdTech must address more than intuitive interfaces. It must account for the student’s sense of agency, the teacher’s trust in the system, and the institution’s ethical responsibility. As Norman (2004, p. 143) reminds us, “Trust is an essential ingredient in cooperative, human interaction.” Yet, many current platforms prioritize administrative convenience or business scalability over emotional resonance or pedagogical sensitivity, neglecting the human relationships at the core of education.
Consider the difference between a teacher who adjusts their lesson on the fly based on student expressions and an AI system that recommends the next module based on click patterns. One perceives the learner as a whole person; the other as a datapoint. The risk is not just poor design, it’s dehumanization.
Designing AI That Understands Humans
We can adopt human-centered design principles and apply them to AI systems to ensure AI supports the human experience. This may include:
Transparency: AI systems should explain their decisions in language that users can understand.
Empathy: Models must be trained not just on large datasets but on idiosyncratic and contextually rich examples of human behavior.
Customizability: Users should be able to adjust AI behavior to suit their preferences and values.
Responsiveness: Systems should learn from user feedback, not just data trends.
Most importantly, AI must be seen not as a replacement for human judgment but as an augmentation of it. In education, this could mean designing AI tools that aim to support teachers and learners at a deeper level instead of reducing them to roles in a feedback loop.
Lessons from Japan’s Human-Centered Systems
Returning to my experience in Japan, I am reminded that thoughtful design is not just about cutting-edge technology; it’s about intentionality. The systems in Tokyo did not merely function; they resonated. The user-first mindset was not a slogan, but a practice. Designers, engineers, and policymakers had clearly asked not just “What is possible?” but “What is humane?”
This ethos should be the foundation of how we build AI systems for education. We must ask: Does this tool respect the learner’s dignity? Does it honor the teacher’s role? Does it serve the institution’s mission to foster growth, not just metrics?
The Road Ahead: A Call for Human-AI Partnership
As we march forward into an AI-enhanced future, we must remember that progress is not just about automation but about elevation. The tools we create should lift the human spirit, not flatten it. This is especially crucial in education, where every system carries moral influence. We are not just optimizing workflows; we are influencing minds.
AI has the potential to be more than just efficient; it can be attentive, supportive, and responsive to the human experience. Rather than focusing solely on logic, we might also consider how these systems can be designed with empathy in mind.
Reference:
Norman, D. A. (2004). Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books.
Please cite the content of this blog:
Correia, A.-P. (2025, April 25). The Human Experience: Are we losing the user-first mindset as AI systems assume greater roles in our daily lives? Ana-Paula Correia’s Blog. https://www.ana-paulacorreia.com/blog/the-human-experience-are-we-losing-the-user-first-mindset-as-ai-systems-assume-greater-roles-in-our-daily-lives