The demo looked great. Six months later, half your teachers still haven’t logged in, and the renewal invoice just landed. If you want to know how to evaluate an edtech tool before that happens, you’ll need more than a sales call and a free trial.
You need a short, honest process. Here’s one built on current district usage data, the federal rules that changed in 2026, and what the research actually says about AI in classrooms.
Why So Many EdTech Purchases Quietly Fail
The numbers are hard to ignore. Districts accessed an average of 2,982 distinct edtech tools in the 2024 to 2025 school year, nearly 9% more than the year before. Instructure’s 2026 report put access at about 3,001 tools, yet its own Canvas data showed students and educators launching an average of just four integrated tools a year.
Access isn’t used. Not even close.
Older audits tell the same story. A University of Pennsylvania analysis of BrightBytes data from 48 districts found a median of only 30% of software licenses got used, and 97.6% never saw intense use (at least 10 hours between assessments). Glimpse K12 put the share of unused licenses at 67%.
Quality is thin at the top, too. Of the 40 most used tools in that 2026 report, only two (Canvas LMS and Newsela) met all five quality standards a national review checks for: research, accessibility, interoperability, data privacy, and usability.
So learning about the evaluation of an edtech tool isn’t busywork. It’s how you stop paying for software nobody opens. Our breakdown of the biggest edtech failures shows how expensive that gets.
Check 1: Write Down the Problem Before You Watch a Demo
Start with one sentence. Something like: “Forty percent of our 7th graders read two or more grade levels behind, and teachers have no fast way to group them.”
That sentence gives you a target group and something to measure. Pull your baseline now (benchmark scores, assignment completion, hours spent grading) so you have something to compare against later.
Then ask the awkward question. Do you already own a tool that does this? With roughly 3,000 tools floating around a typical district, the answer is often yes.
Most teams skip this step, and that’s when the whole question of how to evaluate an edtech tool shrinks down to “which demo felt more fun.” If you want a ready script for this stage, use our list of edtech evaluation questions.
Check 2: Ask for Evidence That Matches Your Students
Every vendor says its product works. Ask how they know.
The Every Student Succeeds Act (ESSA) sorts evidence into four tiers:
- Tier 1, Strong: at least one well-designed randomized experiment.
- Tier 2, Moderate: at least one matched comparison study.
- Tier 3, Promising: a correlational study with statistical controls.
- Tier 4, Demonstrates a Rationale: a research-based logic model, plus a plan to study it.
Here’s what vendors gloss over. Tier 4 only means the company has a theory and a plan to test it. In Instructure’s 2026 report, 52.5% of the top 40 tools had at least Tier 4 evidence, which sounds better than it is. And if you’re spending Title I school improvement money, the tool usually needs Tier 1, 2, or 3.
Push further. Was the study run with students like yours (same grades, similar reading levels, a similar share of English learners)? How many minutes a week did kids need to see results?
That dosage question exposes more weak claims than anything else in evaluating an edtech tool. If the study needed 90 minutes a week and your schedule allows 30, it doesn’t apply to you.
Check 3: Judge Classroom Fit With SAMR and Triple E
Evidence says a tool can work. It doesn’t say your teachers will use it well, and that’s where evaluating technology use in the classroom comes in.
SAMR, created by Dr. Ruben Puentedura, sorts tech use into four levels. Here are SAMR model examples built around one assignment, a persuasive essay:
| SAMR level | What it looks like with a persuasive essay |
|---|---|
| Substitution | Students type the essay in Google Docs instead of writing by hand. Nothing else changes. |
| Augmentation | They use voice typing and spell check, and the teacher leaves margin comments. |
| Modification | Classmates give live feedback in shared comments, and students revise using version history. |
| Redefinition | Students record the argument as a podcast and send it to the city council for a real reply. |
Don’t treat SAMR as a ladder you must climb. Substitution is perfectly fine for a spelling quiz.
Triple E, developed by Liz Kolb at the University of Michigan, asks whether a tool engages students in the learning goal, enhances it, and extends it beyond class. A common example: playing a Kahoot game is engagement, but building one is enhancement.
Use both during demos. If a vendor can only show you substitution, you’re paying for a digital worksheet (and that’s a fast shortcut in how to evaluate an edtech tool when you don’t have a research team).
Check 4: Run the Privacy and Accessibility Review
This step has legal teeth, and the rules shifted this year.
Privacy. The FTC’s amended COPPA Rule, which protects kids under 13, hit its full compliance date on April 22, 2026. It now counts biometric identifiers (think face and voice data) as personal information and requires a written data retention policy. Ask every vendor:
- Will you sign the National Data Privacy Agreement (NDPA) from the Student Data Privacy Consortium?
- Do you hold a 1EdTech or iKeepSafe privacy certification? Only 45% of the top 40 tools did in 2026.
- Do you use student data to train AI models?
- How fast do you delete our data when the contract ends?
Not a satisfactory answer? Walk away.
Under the Justice Department’s ADA Title II rule, public schools’ web content and apps must meet WCAG 2.1 Level AA. In April 2026, the DOJ pushed the deadlines back a year: April 26, 2027 for entities serving 50,000 or more people, and April 26, 2028 for smaller ones. The duty didn’t disappear, and it covers apps vendors run on a school’s behalf.
Ask for an Accessibility Conformance Report (often called a VPAT), then test it yourself. Tab through the app without a mouse. Turn on a screen reader.
Five minutes tells you a lot. Skip this part of evaluating an edtech tool, and a $5,000 app can turn into a legal problem.
If the Tool Uses AI, Add These Checks
AI now shows up in almost everything, and teachers aren’t waiting. A RAND survey found 53% of core subject teachers used AI for instruction by 2025, but only 45% of principals said their school or district had an AI policy.
The research on AI tools for personalized learning is promising and messy at the same time:
- A 2025 Harvard trial with 194 physics students found a custom AI tutor produced more than double the learning gains of an active learning class in less time. That tutor was built to guide students through problems, not hand over answers.
- A 2025 study in PNAS with about 1,000 high school math students found the flip side. Kids using a basic GPT tutor did better during practice, then scored 17% worse on the exam once the AI was gone.
Same technology. Opposite results.
That’s why the best teaching AI tools aren’t the flashiest ones. In August 2025, Common Sense Media rated AI teacher assistants (including Gemini in Google Classroom, Khanmigo, Curipod, and MagicSchool) as moderate risk, mostly over bias, accuracy, and teachers pushing unreviewed content straight to students.
For AI products, evaluating an edtech tool comes down to four extra questions:
- Does it give hints and ask questions, or just hand over answers?
- Can teachers see student chats and switch features off?
- Does AI output get teacher review before students see it?
- Is student data kept out of model training?
Check 5: Pilot It Like You Mean It
A free trial isn’t a pilot. A pilot has a question, a timeline, and a finish line.
Pick two to four classrooms and run the tool for six to eight weeks, ideally next to similar classes that don’t use it. Track four things:
- Usage: weekly minutes per student, compared with the dosage the vendor’s research required.
- Outcome: the baseline metric you set in Check 1.
- Teacher time: hours saved or added each week.
- Fit: whether it works with your login system (Clever, ClassLink, or Google), roster syncing through OneRoster, and your LMS through LTI.
That last one matters more than people think. A tool that makes teachers manage a separate login and class list will lose users by week three. Our roundup of edtech tools that fit smoothly into teacher workflows shows what good integration looks like.
Collect teacher and student feedback too, but weigh it against the numbers. Kids love plenty of things that don’t teach them anything. The pilot is the stage of how to evaluate an edtech tool, where opinions finally meet data.
Check 6: Count the Real Cost, Not the Sticker Price
Licenses are often the cheap part. Here’s a simple example for one middle school:
| Cost item | Year 1 example |
|---|---|
| License: $8 per student × 600 students | $4,800 |
| Training: 25 teachers × 6 hours × $40 an hour | $6,000 |
| IT setup and rostering: 20 hours × $50 an hour | $1,000 |
| Total | $11,800 |
Training costs more than the software. That’s not unusual.
Now ask about year two. Does the price jump after an intro rate? Is there a seat minimum? Can you export your data in a standard format if you leave? Building those answers into how to evaluate an edtech tool keeps renewals from turning into surprises. (On the vendor side? Our guide on how to start an edtech startup covers what buyers like this expect.)
Final Thoughts
Knowing how to evaluate an edtech tool really comes down to one habit: name the problem first, then make every product prove it can solve it for your students, safely, at a cost you can keep paying.
Score each check from 1 to 3. Set one hard rule, too: a 1 on privacy or accessibility is an automatic no, no matter how much teachers liked the demo.
Your next step is simple. Pick one tool that’s up for renewal this semester and run it through all six checks. You might find you don’t need it at all.
Frequently Asked Questions (FAQs)
Short answers to what people ask most about how to evaluate an edtech tool.
What are the five criteria for evaluating educational technology?
A widely used set covers research evidence, accessibility, interoperability, data privacy, and usability. Only 2 of the 40 most used K12 tools met all five in Instructure’s 2026 report.
What is the SAMR model, with an example?
SAMR stands for Substitution, Augmentation, Modification, and Redefinition. Typing an essay instead of handwriting it is Substitution. Publishing that essay as a podcast for a real audience is Redefinition.
How long should an edtech pilot last?
Six to eight weeks. Shorter pilots mostly measure novelty.
Are free edtech tools safe for students?
Not automatically. Free tools often earn money from data, so read the privacy policy, ask whether student data trains AI models, and look for a signed NDPA before any student logs in.
How do I know if an AI tool is good for students?
Look for guardrails. Good tools coach with hints instead of giving answers, let teachers see and control what students get, and keep student data out of model training. Then ask for results from real classrooms, not just a feature list.
What questions should I ask an edtech vendor?
Ask what problem it solves, which ESSA tier backs it, how many weekly minutes the research required, whether they’ll sign the NDPA, and what year two will cost.
