Where does a photo go when you upload it?
A photo app does not keep your picture on one phone. It sends it to a storage service in the cloud, puts a note on a queue so another program can shrink it, and records it in a table. Those three jobs are what S3, SQS and DynamoDB do.
By the end of this lesson
Note
I can use the AWS command line to store, queue and look up data in a practice cloud.
Key words
- Bucket: a named container for files in S3.
- Object: one stored file, found by its key.
- Queue: messages waiting in order to be handled.
- Table: items stored under a key, in DynamoDB.
- Emulator: a copy that behaves like a service but is not the real one.
Starter: what you already know
Note
Answer in your head. 1. What is a file name? 2. What is a queue at a bus stop for? 3. What is one row of a database table? Answers: the name you find the file by; so people are served in order; one record.
Before you start
Note
Press Start the lab on the bar at the top of this lesson and wait until it says it is running. The stack is a service called cloud, and the terminal reaches it by that name. Only S3, SQS and DynamoDB are switched on here: ask for any other service, Lambda for one, and you get an error. What you make is kept when the lab is stopped and started again. Each Run button below types its command into your lab's terminal, one after another in the same session. To watch them, or to type your own, press Open terminal on the bar.
1. A bucket holds objects
The first command gives the aws command a made-up key, since the stack checks none, the region us-east-1 and the address of your own stack, and prints nothing. mb makes a bucket and prints make_bucket: lab-bucket. The next sends the text hello cloud to it under the key hello.txt and prints nothing. Copying to - prints the object: hello cloud. The last asks for a key that was never stored. The reply is an error that says NoSuchKey and The specified key does not exist., which is the same on real AWS.
export AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_DEFAULT_REGION=us-east-1 AWS_ENDPOINT_URL=http://cloud:4566Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws s3 mb s3://lab-bucketShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
echo "hello cloud" | aws s3 cp - s3://lab-bucket/hello.txtShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws s3 cp s3://lab-bucket/hello.txt -Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws s3api get-object --bucket lab-bucket --key nothing.txt /dev/nullShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
2. A queue holds messages until they are deleted
The first command makes a queue and keeps its address in Q, so you never type it. The second sends a message and prints an MD5OfBody and a MessageId. The third receives it, prints it under Messages with its Body, first message, and saves the reply in reply.json. The fourth cuts out its ReceiptHandle, the proof you received it, and keeps it in H. delete-message prints nothing. Many people think receiving takes a message off the queue. It only hides it for a while, and it comes back and is handled twice unless you delete it.
Q=$(aws sqs create-queue --queue-name lab-queue --query QueueUrl --output text)Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws sqs send-message --queue-url $Q --message-body "first message"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws sqs receive-message --queue-url $Q | tee reply.jsonShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
H=$(grep -o '"ReceiptHandle": "[^"]*"' reply.json | cut -d'"' -f4)Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws sqs delete-message --queue-url $Q --receipt-handle "$H"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
3. A table holds items under a key
The first command makes a table called Books, whose key is an author and a title, and prints "ACTIVE". The second puts two items in one request and prints an empty UnprocessedItems. get-item asks for one item by its whole key and prints it, every value wrapped in its type: S for text and N for a number. query reads every item under one author and prints Count 2 and both titles. Ask get-item for a key you never put and it prints nothing: here a missing item is an empty reply, not an error.
aws dynamodb create-table --table-name Books --attribute-definitions AttributeName=author,AttributeType=S AttributeName=title,AttributeType=S --key-schema AttributeName=author,KeyType=HASH AttributeName=title,KeyType=RANGE --billing-mode PAY_PER_REQUEST --query TableDescription.TableStatusShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws dynamodb batch-write-item --request-items '{"Books":[{"PutRequest":{"Item":{"author":{"S":"Ada"},"title":{"S":"Notes"},"year":{"N":"1843"}}}},{"PutRequest":{"Item":{"author":{"S":"Ada"},"title":{"S":"Sketch"},"year":{"N":"1842"}}}}]}'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws dynamodb get-item --table-name Books --key '{"author":{"S":"Ada"},"title":{"S":"Notes"}}'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
aws dynamodb query --table-name Books --key-condition-expression "author = :a" --expression-attribute-values '{":a":{"S":"Ada"}}' --query '{Count:Count,Titles:Items[].title.S}'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Exit check
Tip
Answer without looking back. 1. What finds an object in a bucket? 2. What error does S3 give for a key that is not there? 3. What must you do to a message after you have handled it? 4. Is this stack the real AWS? Answers: its key; NoSuchKey; delete it; no, it is an emulator, a practice copy.
For teachers
Tip
This lesson is a small example of a lesson worked in a lab: the steps and the Run buttons are the lesson, and the lab is each student's own copy. The lab is one small service and one named volume, and nothing here marks itself: the student hands in the replies of their own session, a bucket with an object put, listed and got, a queue with a message sent, received and deleted, a table with an item put, got and queried, and the error for a key that is not there, with a sentence on what MiniStack is not. Key learning points: 1. S3 stores objects under keys in a bucket. 2. A queue holds messages, and a message is finished by deleting it. 3. DynamoDB stores items under a key, and a query reads every item under one part of it. 4. A missing S3 key is an error and a missing DynamoDB item is an empty reply. 5. MiniStack is an emulator, not AWS. Common mistake: thinking receive removes a message. Equipment: the lab and its terminal. Tip: ask students to predict what each reply will say before they press Run. Lambda is switched off on purpose, since its functions would run student code inside the stack. Message ids, receipt handles, times and the queue address differ from real AWS and from run to run. Copy it into your own module and change it, or build one from scratch with the Lab menu in the lesson builder.