Beyond AI Rules: Building Human Responsibility into Education and Software

Generative AI is forcing universities and businesses to reconsider how they govern technology that is developing faster than many of their existing policies. As both a college lecturer and a software manager, I encounter this challenge from two perspectives. In both environments, I believe the debate needs to move beyond a simple choice between banning AI and approving a limited set of tools.

The more important question is how organizations create the conditions for responsible use. That requires clear rules, but also something more fundamental: people must remain accountable for how these systems are used, what information is provided to them, and how their outputs are interpreted.

How should universities protect student data when using AI?

Universities are already taking very different approaches. Boise State University, for example, follows a centralized model in which students and staff are expected to use approved enterprise AI tools. Stanford takes a more contextual approach, treating AI more like a peer tutor and restricting its use for activities such as examinations or essay writing unless explicitly permitted.

These differences reflect a broader tension within higher education. University leadership often sees generative AI as an opportunity to modernize education and prepare students for a changing labour market. Faculty, on the other hand, are more concerned about plagiarism, hallucinations and the additional effort required to redesign courses and assessment. A global AACSB survey illustrates this difference in perspective.

At the same time, the discussion should not focus exclusively on risk. A study involving more than 400 university professors in Spain found that usefulness and ease of use were important drivers of AI adoption. It also reported positive effects on teacher well-being, including higher energy levels and slightly lower job stress, partly because AI helped reduce time spent on administrative tasks.

This is precisely why blanket restrictions are insufficient. AI can provide meaningful benefits, but those benefits need to be balanced against privacy, reliability and accountability.

Data protection is a good example. It is too simplistic to assume that every AI system automatically learns from everything a user enters. Data handling depends on the provider, product, configuration and contractual arrangements. Responsible governance therefore requires universities to understand how a specific service processes and retains information before deciding what data may safely be used.

Why are policies and AI detectors not enough?

Many universities respond to these concerns through approved-tool policies, enterprise agreements and contractual safeguards with technology providers. These measures are necessary, but they should be regarded as the starting point of governance rather than its end.

One of the practical challenges is that policies can become so legalistic or technical that the people expected to follow them do not know how to translate them into everyday behavior. Current university guidelines are often criticized for being too vague or for burying actionable rules beneath legal and technical language.

Effective AI governance should therefore answer practical questions clearly. What information may I enter into this system? How should I validate its output? When is human review required? Who is responsible for the final result? And what happens when the system produces an incorrect or inappropriate outcome?

These questions cannot be solved simply by placing a tool on an approved list.

The same applies to automated AI detection. Universities are increasingly moving away from free web-based AI detectors because of their unreliability and frequent false positives. Uploading student work to unvetted third-party platforms may also introduce an additional privacy risk.

Replacing human judgement with another imperfect automated system is not effective governance.

This connects directly with how I approach AI more generally. Fluency is not evidence of truth. An AI system can produce an answer that is polished, persuasive and professionally written while still being wrong. In my own work and teaching, that leads to a simple principle: I trust AI for speed, but never for responsibility.

What can businesses learn from universities?

The governance questions universities are confronting are equally relevant to software companies. I see a particularly strong parallel in my own experience developing software for the justice sector, where digital systems support complex and legally significant processes.

In such an environment, technological efficiency cannot be considered in isolation. Reliability, legal validity, security, traceability, accessibility and human accountability are equally important. The objective is not to exclude AI from these environments, nor to delegate consequential decisions to machines. Rather, any AI-enabled functionality operating close to legally significant workflows must be introduced with an appropriate level of scrutiny.

The university examples offer several useful lessons for software organizations. Governance needs to involve different disciplines rather than being left exclusively to technical teams. Users need practical AI literacy. Acceptable and unacceptable uses must be explicit. Decisions, data flows and limitations should be documented. Systems must be tested in the environment in which they will actually operate, and their performance and risks must continue to be evaluated after deployment.

For software companies, responsible AI is therefore not a choice between innovation and regulation. Good governance, security, risk management, testing, documentation and human oversight should be integrated throughout design, development, deployment and support.

Universities are already showing both the opportunities and the difficulties of putting these principles into practice. The broader lesson is clear: trustworthy AI will not come from rules alone. It will depend on whether organizations combine innovation with professional judgement, transparent governance and people who remain accountable for the consequences of the technology they use.