Is the app any good?
Your team finishes a quiz app. You think it is great. But what do the users think?
You could ask them. You could watch them. You could count how often they use it.
Each way gives you a different kind of answer. A good developer knows many ways, and chooses the right one.
Key words
| Word | What it means | Example |
|---|---|---|
| Qualitative data | Data in words: opinions, feelings and reasons. | "The menu is confusing." |
| Quantitative data | Data in numbers: counts, times and scores. | 4 out of 5 stars; 12 seconds |
| Survey | A set of questions that many people answer. | an online form |
| Interview | A planned conversation with one person. | ten minutes with a teacher |
| Focus group | A small group of users who talk about a product together. | six students and a leader |
| Observation | Watching people use the product and noting what happens. | seeing a user get stuck |
| Performance data | Numbers about how the software runs. | page load time |
| Use data | Numbers about how people use the software. | how many users press Pay |
| Appraisal | A planned meeting where your manager looks at your work and gives feedback. | a yearly review with your line manager |
| Peer mentoring | A person at a similar level helps and advises you. | a classmate reviews your code |
The cells students fill in are left blank in this preview.
Two kinds of data
| Question | Qualitative | Quantitative |
|---|---|---|
| Form | Words, opinions, stories | Numbers |
| Answers the question | Why? How do you feel? | How many? How long? How much? |
| Example | "I could not find the Pay button." | 14 of 20 users could not find the Pay button. |
| Good for | Understanding reasons | Seeing the size of a problem |
| Weakness | Hard to count, takes time | Does not say why |
The cells students fill in are left blank in this preview.
Quick check: words or numbers?
Have a go
Quick check: words or numbers?
8 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
Eight methods
| Method | What you do | Kind of data |
|---|---|---|
| Verbal feedback (informal) | Chat with a user and listen | Qualitative |
| Verbal feedback (formal) | A planned meeting, with notes | Qualitative |
| Survey or questionnaire | Many people answer the same questions | Both: ratings (numbers) and comments (words) |
| Performance and use data | Read logs and counters | Quantitative |
| Observation | Watch users, fill in a record | Both: times (numbers) and behaviour (words) |
| Focus group | Talk with a group that is like the users | Qualitative |
| Interview | Ask one person many questions | Qualitative |
| Peer mentoring, line management, appraisal | A peer or your manager gives feedback on your work | Mostly qualitative |
The cells students fill in are left blank in this preview.
Try it: read some performance data
load_times = [2, 3, 2, 8, 3, 2, 4, 9, 3, 2] # seconds
total = sum(load_times)
average = total / len(load_times)
slowest = max(load_times)
slow_pages = 0
for t in load_times:
if t > 5:
slow_pages = slow_pages + 1
print("Average:", average)
print("Slowest:", slowest)
print("Pages slower than 5 seconds:", slow_pages)
# Change a number and run it again.
Observation: watch and write
In an observation you watch a user try the app. You say nothing. You do not help.
You fill in an observation record. It says the task, the time taken, what went wrong, and what the user said. Watching shows problems that people forget to tell you about.
An observation record
| Task | Time | Problems seen | What the user said |
|---|---|---|---|
| Find the homework page | 40 seconds | Pressed Messages by mistake | "Where is it?" |
| Hand in work | 1 minute 10 seconds | Did not see the Upload button | "Oh, it is there." |
| Log out | 5 seconds | None | Nothing |
The cells students fill in are left blank in this preview.