Lab — Latency, sizing and the edge
Time: 120 minutes — 10 setup, 80 core, 20 evidence, 10 teardown.
Prerequisite: the S1 account and project, the course-s1-<id> SSH key and the root domain you registered in S1 self-study. The Cloudflare zone must be Active before this lab, because step 3 puts s2.<root> behind the edge.
You need to know: latency is time to the first byte; bandwidth is bytes per second; a faster link does not shorten the wait before the first byte. A p95 is the value 95% of requests finish within, so it describes the tail rather than the average. DigitalOcean has no region in Thailand, so the origin is in Singapore (SGP1) and Cloudflare's edge carries the request from Chiang Mai. A Basic Droplet shares its vCPU with other customers and has no CPU-credit balance; benchmark it to learn what it delivers. Cloudflare caches a .jpg by default and does not cache .html by default. SSL mode Full encrypts the edge-to-origin leg but does not validate the origin certificate.
Safe-data acknowledgement: I have read the lab safe-data policy. I will use only the course-supplied synthetic media and will not upload real personal media to any cloud service.
Task
Put one 1 GB Rocky Linux 10 Droplet in SGP1 behind a proxied s2.<root> record with SSL Full. Measure the same JPEG from the origin and through the edge, watch it go MISS → HIT while the HTML control stays uncached, benchmark the CPU and disk, exhaust the instance's memory, resize it to 2 vCPU / 2 GB without growing the disk, and rerun.
Work from the downloaded week folder. Commands marked Laptop run on your machine; commands marked Droplet run after you SSH in. Panel paths sit beside the CLI step where both exist. A step with no CLI path says so.
Steps
1. Pre-flight and token regeneration
Start with the zone and the account, then regenerate the S1 token. The S1 token cannot be edited, so the resize later would be refused with a 403. That refusal is the lesson: create a new token with the extra scopes and delete the old one.
Panel: Cloudflare dashboard → your zone → check Active. DigitalOcean Control Panel → API → Tokens & Keys → Generate New Token.
Create course-s2-<id> with the S1 scopes plus two more:
| Scope group | Scopes |
|---|---|
| Droplet | droplet:read, droplet:create, droplet:delete, droplet:update |
| SSH key | ssh_key:read, ssh_key:create |
| Firewall | firewall:read, firewall:create, firewall:update, firewall:delete |
| Project | project:read, project:create, project:assign_resource |
| Tag | tag:read, tag:create |
| Account and reads | account:read, regions:read, sizes:read, image:read, billing:read |
Set the expiry to 90 days and copy the secret once. If doctl does not recognise the new token, add it as a context; the macOS config path differs from Linux.
Laptop:
export DIGITALOCEAN_ACCESS_TOKEN=<new-token>
doctl account get
doctl account get proves the token works and returns the account status. The token value is a credential: never paste it into a file that enters the repository, and keep the config at chmod 600.
Laptop:
dig NS <root> +short
doctl projects list --format ID,Name
doctl compute ssh-key list --format FingerPrint,Name
dig NS must return exactly two Cloudflare nameservers. doctl projects list gives the <project-uuid> to attach the Droplet to; doctl compute ssh-key list gives the <fingerprint> of the course-s1-<id> key you created in S1.
Panel: delete the old course-s1-<id> token. The new token keeps the same least privilege plus the resize scopes.
2. Create the week-2 firewall
S1's firewall is attached by the tag week:1; this lab owns a separate rule set, so a changed laptop IP does not depend on the old rule.
Laptop:
doctl compute tag create "week:2"
A cloud firewall can only reference a tag that already exists, so create the tag first. Step 3's Droplet carries it, and the firewall applies as soon as the tag is attached.
Panel: Manage → Tags → Create Tag week:2, then Networking → Firewalls → Create Firewall — name course-<id>-s2, inbound TCP 22 from your current IP only and TCP 443 from anywhere, outbound all, applied to the tag week:2.
Laptop:
MYIP=$(curl -s https://api.ipify.org)
doctl compute firewall create --name "course-<id>-s2" \
--inbound-rules "protocol:tcp,ports:22,address:$MYIP/32 protocol:tcp,ports:443,address:0.0.0.0/0" \
--outbound-rules "protocol:tcp,ports:0,address:0.0.0.0/0 protocol:udp,ports:0,address:0.0.0.0/0 protocol:icmp,ports:0,address:0.0.0.0/0" \
--tag-names "week:2"
MYIP resolves your public address, so SSH is open only to you. Port 443 is open to everyone because Cloudflare's edge connects from many addresses, not one. The output is the <firewall-id>.
Laptop:
doctl compute firewall get <firewall-id> -o json > data/firewall-rules.json
Save the rule set as evidence: the inbound SSH source and the 443 rule are what the marker checks.
3. Create the Droplet
Panel: Droplets → Create Droplet — course-<id>-s2, region SGP1, image Rocky Linux 10 x64, plan Basic / 1 vCPU / 1 GB / 25 GiB, the course-s1-<id> SSH key, tags course:42-05-0112, student:<id>, week:2, project course-<id>.
Laptop:
doctl compute droplet create "course-<id>-s2" \
--region sgp1 --image rockylinux-10-x64 --size s-1vcpu-1gb \
--ssh-keys <fingerprint> \
--tag-names "course:42-05-0112,student:<id>,week:2" \
--project-id <project-uuid> \
--wait
--region sgp1 is DigitalOcean's nearest region to Thailand; there is no Thailand region. --wait returns when the Droplet is active. The command prints the <droplet-id>.
Laptop:
doctl compute droplet list -o json > data/droplet-before.json
doctl compute droplet get <droplet-id> --format PublicIPv4 --no-header
The JSON is the "before" inventory. The second command prints the <droplet-ip> you use for SSH and for the origin leg of the latency test.
4. Add the proxied record
Panel: Cloudflare dashboard → DNS → Records → Add record — type A, name s2, IPv4 address <droplet-ip>, proxy status Proxied, TTL Auto. There is no CLI path in this lab; the API-token path is an extension in Session 4.
Laptop:
dig +short s2.<root>
The answer must be Cloudflare anycast addresses, not <droplet-ip>. If it returns the Droplet address, the record is not proxied: edit it and switch the orange cloud on. Propagation is usually seconds.
API-token extension (reference runner only). The panel path is the student path. The reference runner has no browser, so it creates the same record with a scoped Cloudflare token and records the API call in its command log.
curl -s -X POST -H "Authorization: Bearer <cloudflare-token>" -H 'Content-Type: application/json' \
-d '{"type":"A","name":"s2","content":"<droplet-ip>","ttl":1,"proxied":true}' \
"https://api.cloudflare.com/client/v4/zones/<zone-id>/dns_records"
The response contains the <record-id> used by the teardown. The token needs Zone:DNS:Edit; it never appears in the command log.
5. Build the HTTPS origin
Laptop:
ssh -i ~/.ssh/do_s1 root@<droplet-ip>
Rocky's cloud image accepts the key only; the login is root.
Droplet:
dnf install -y python3 openssl
Install the interpreter and the certificate tool. The cloud image usually carries both; the command is idempotent.
Droplet:
systemctl is-active firewalld && { firewall-cmd --permanent --add-service=https; firewall-cmd --reload; } || true
If Rocky's host firewall is active, open 443 inside the VM as well. The DigitalOcean cloud firewall is the outer filter; this is the inner one. A host with firewalld blocking 443 returns 522 at the edge.
Droplet:
mkdir -p /srv/s2 && cd /srv/s2
cat > index.html <<'HTML'
<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>S2 origin</title></head>
<body><h1>S2 origin</h1></body>
</html>
HTML
Create the served directory and the HTML control. It has no cache-friendly extension, so Cloudflare will not cache it by default; it is the comparison for the JPEG.
Droplet:
openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem \
-days 2 -subj "/CN=s2.<root>"
Generate a self-signed certificate for the hostname. Cloudflare's Full mode accepts it because Full encrypts but does not validate; Full (strict) would reject it. The certificate is temporary by design.
Laptop (open a second terminal, stay in the week folder):
scp -i ~/.ssh/do_s1 data/thumb-sample.jpg root@<droplet-ip>:/srv/s2/thumb-<id>.jpg
scp -i ~/.ssh/do_s1 serve.py root@<droplet-ip>:/srv/s2/serve.py
Upload the synthetic JPEG under a unique name and the origin server. The JPEG is the cacheable object; the unique name keeps your run out of another student's edge copy.
Droplet:
cd /srv/s2
nohup python3 serve.py --port 443 --dir /srv/s2 --cert cert.pem --key key.pem > serve.log 2>&1 &
sleep 2
curl -sk https://127.0.0.1/thumb-<id>.jpg -o /dev/null -w '%{http_code}\n'
Start the HTTPS server on 443. The curl uses -k because the certificate is self-signed; it must print 200. A 000 means the server is not running; read serve.log.
Droplet:
nproc; lscpu | grep -E 'Model name|^CPU\(s\)'; free -h; df -h /
Record the "before" system facts: one vCPU, about 1 GiB of memory, a 25 GiB disk. These lines belong in the benchmark evidence.
6. Set SSL Full and observe the cache
Panel: Cloudflare dashboard → SSL/TLS → Overview → select Full. There is no CLI path in this lab.
API-token extension (reference runner only). The runner sets and later restores the same setting through the zone API:
curl -s -X PATCH -H "Authorization: Bearer <cloudflare-token>" -H 'Content-Type: application/json' \
-d '{"value":"full"}' "https://api.cloudflare.com/client/v4/zones/<zone-id>/settings/ssl"
The token needs Zone:Zone Settings:Edit. The runner records the previous mode and restores it at teardown, because a shared demo zone may serve other hostnames.
Laptop:
curl -sD - -o /dev/null "https://s2.<root>/thumb-<id>.jpg" | grep -i cf-cache-status
The first request arrives at the nearest colo, misses, fetches the origin and stores the JPEG. Expect cf-cache-status: MISS and HTTP 200. A 522 means the edge could not reach the origin: check the Droplet's server and its host firewall.
Laptop:
curl -sD - -o /dev/null "https://s2.<root>/thumb-<id>.jpg" | grep -i cf-cache-status
The second request is served from the colo. Expect cf-cache-status: HIT. The default edge TTL for a JPEG is two hours.
Laptop:
curl -sD - -o /dev/null "https://s2.<root>/index.html" | grep -i cf-cache-status
The HTML control has no HIT: Cloudflare does not cache .html by default, so this request reaches the origin every time. A missing header is the expected result, not an error.
Laptop:
curl -s "https://s2.<root>/cdn-cgi/trace" | grep -E 'colo|loc'
/cdn-cgi/trace is answered by Cloudflare itself. colo= names the PoP that served you — CNX, BKK or SIN in this region — and loc= the country Cloudflare inferred. Cloudflare chooses by routing, not by geography, so record what you actually got.
7. Measure both paths
Laptop:
python3 latency.py --origin "https://<droplet-ip>/thumb-<id>.jpg" \
--edge "https://s2.<root>/thumb-<id>.jpg" \
--samples 10 --insecure-origin --warm-edge > data/latency_measured.csv
head -8 data/latency_measured.csv
The origin path connects directly to Singapore and uses --insecure-origin because of the self-signed certificate. The edge path connects to the colo. Each row splits the request into DNS, TCP connect, TLS and time to first byte; total_ms is the wall time. Ten samples per path are the minimum for a p95.
--warm-edge adds a third block, cloudflare-edge-warm: ten requests on one kept-alive connection. A browser reuses a TLS connection after the first request, so this block is the viewer's steady state. The cloudflare-edge block is the cold start, and the difference is the handshake and connection cost that a cache cannot remove. On the reference network the cold edge p95 sat near 70 ms and the warm p95 near 15 ms; both are below the 145–150 ms origin path. The warm block is the one that can meet the 30 ms target; the cold block pays the handshake that a cache cannot remove.
8. Benchmark the instance
Droplet:
cd /srv/s2
python3 bench.py --seconds 5 --workers 1
python3 bench.py --seconds 5 --workers 2
The first row is one worker on one vCPU; the second is two workers. On s-1vcpu-1gb the two workers share the advertised vCPU, so the parallel rate usually stays close to the single rate; a shared host can also burst above it. After the resize the second vCPU may or may not add a clean multiplier. Record what you get and explain it; the resize's hard proof is nproc, free -h and the memory recovery, not a CPU ratio. Copy both CSV rows into data/benchmark_measured.csv.
Droplet:
dd if=/dev/zero of=/root/testfile bs=1M count=1024 oflag=direct
dd if=/root/testfile of=/dev/null bs=1M iflag=direct
rm -f /root/testfile
Write and read a 1 GiB file with direct I/O. oflag=direct and iflag=direct bypass the page cache, so the MB/s figures describe the disk instead of RAM. Without them the read measures memory and looks unrealistically fast. Record the MB/s lines: this is the disk baseline the resize must leave unchanged.
9. Exhaust the memory
Droplet:
free -h
swapon --show
Confirm the plan has about 1 GiB and no swap. With no swap, the ceiling is hard.
Droplet:
python3 - <<'PY'
try:
block = bytearray(1500 * 1024 * 1024)
print("allocated", len(block))
except MemoryError:
print("MemoryError")
PY
echo "exit=$?"
dmesg | tail -5
Ask for 1.5 GiB on a 1 GiB machine. Python raises MemoryError or the kernel kills the process; either result is the deliberate failure. dmesg shows the out-of-memory line when the kernel ends the process. Save the output.
10. Resize without growing the disk
Panel: Droplets → course-<id>-s2 → Resize → CPU and RAM only → s-2vcpu-2gb. Do not tick "Resize disk". There is also a CLI path.
Laptop:
doctl compute droplet-action resize <droplet-id> --size s-2vcpu-2gb --resize-disk=false --wait
--resize-disk=false keeps the 25 GiB disk and makes the change reversible; a disk can only grow, so growing it is a one-way decision owned by the storage session. The action powers the Droplet off and on. If it comes back off, start it.
Laptop:
doctl compute droplet-action power-on <droplet-id> --wait
Run this only if the resize left the Droplet powered off. --wait returns when the action finishes.
Laptop:
ssh -i ~/.ssh/do_s1 root@<droplet-ip>
Reconnect. The public IP and the disk are unchanged; only vCPU and RAM grew.
11. Rerun and prove the recovery
Droplet:
nproc; free -h; df -h /
python3 - <<'PY'
block = bytearray(1200 * 1024 * 1024)
print("allocated", len(block))
PY
A 1.2 GiB allocation now succeeds. It is deliberately below the 1.5 GiB that failed: a 2 GB plan reports about 1.7 GiB usable after the kernel's own memory, and the running system already holds a few hundred megabytes, so 1.5 GiB can still be refused. The proof is that the ceiling moved and the disk size is still 25 GiB; it is not a claim that every allocation up to 2 GB fits.
Droplet:
cd /srv/s2
nohup python3 serve.py --port 443 --dir /srv/s2 --cert cert.pem --key key.pem >> serve.log 2>&1 &
sleep 2
curl -sk https://127.0.0.1/thumb-<id>.jpg -o /dev/null -w '%{http_code}\n'
The resize powered the Droplet off, so the in-memory server from step 5 stopped. Restart it before rerunning the measurements; the 200 proves it is serving again.
Droplet:
cd /srv/s2
python3 bench.py --seconds 5 --workers 1
python3 bench.py --seconds 5 --workers 2
dd if=/dev/zero of=/root/testfile bs=1M count=1024 oflag=direct
dd if=/root/testfile of=/dev/null bs=1M iflag=direct
rm -f /root/testfile
Rerun the CPU and disk measurements. Single-thread throughput is roughly flat; two-worker throughput should now approach twice the single-worker rate because the second vCPU is usable. Disk throughput stays close.
12. Fill the notebook and assemble the evidence
Put your latency rows in data/latency_measured.csv and your benchmark rows in data/benchmark_measured.csv, then open the lab notebook. The notebook computes p50/p95, marks both paths against the 30 ms target, reads the cache result, checks the resize, and writes latency_results.json. It runs offline on the bundled samples if you use the fallback.
13. Tear down
Laptop:
doctl compute droplet delete <droplet-id> --force
doctl compute firewall delete <firewall-id> --force
Delete both resources. A deleted Droplet stops billing; a detached volume would not.
Laptop:
doctl compute droplet list -o json > data/droplet-after.json
doctl compute firewall list -o json > data/firewall-after.json
doctl balance get
Both lists must be [], and the balance proves the account returned to its starting state. Panel: DNS → delete the s2.<root> record. Keep the project, SSH key, token, zone and spend alert for later sessions.
API-token extension (reference runner only). The runner deletes the record and restores the previous SSL mode:
curl -s -X DELETE -H "Authorization: Bearer <cloudflare-token>" \
"https://api.cloudflare.com/client/v4/zones/<zone-id>/dns_records/<record-id>"
curl -s -X PATCH -H "Authorization: Bearer <cloudflare-token>" -H 'Content-Type: application/json' \
-d '{"value":"strict"}' "https://api.cloudflare.com/client/v4/zones/<zone-id>/settings/ssl"
strict is the value a shared demo zone normally carries. A student zone can keep full or go strict once it has a trusted origin certificate in Session 4.
Evidence bundle
Ten items, scored 3/2/1/0:
data/latency_measured.csvand the p95 chart from the notebook.- The
/cdn-cgi/traceoutput showing the serving colo. - The benchmark table with
lscpu,free -handdf -h, before and after the resize. - The cache headers for the JPEG (MISS then HIT) and the uncached HTML control.
- The memory-failure log and the post-resize success.
- A redacted screenshot of the proxied record and the SSL mode; no token, account ID or email visible.
- The sanitized command log.
- The empty droplet and firewall lists plus
doctl balance get. latency_results.jsonfrom the notebook.- A 150–250-word what this proves note.
Cost ceiling
Under $0.10. The run is one s-1vcpu-1gb hour ($0.00893) and one s-2vcpu-2gb hour ($0.02679) at most; the measurement itself takes minutes. The domain is paid once in S1 and is not part of this ceiling.
Teardown list
- Droplet
course-<id>-s2deleted. - Firewall
course-<id>-s2deleted. - DNS record
s2.<root>removed. - No volume, snapshot or reserved IP created.
- Project, SSH key, scoped token, Cloudflare zone and spend alert kept.
Failure and recovery
| Symptom | Likely cause | Fix |
|---|---|---|
dig +short s2.<root> returns the Droplet IP |
Record is not proxied | Edit the A record and enable the proxy |
curl https://s2.<root>/... returns 522 |
Edge cannot reach the origin | Check serve.py on 443, the Droplet's firewalld, and the cloud firewall's 443 rule |
| The resize returns 403 | The token lacks droplet:update |
Regenerate the token with the S2 scopes; this is the step-1 lesson |
| The second JPEG still says MISS | The cache key changes, or the colo changed | Use the same URL; read cf-ray to see the colo; the edge may not have stored it yet |
MemoryError never appears |
The allocation touched swap or a larger plan | Confirm free -h shows ~1 GiB and swapon --show is empty |
| The two-worker rate does not rise | One vCPU plan, or the workload is serial | Recheck nproc; use Amdahl's law in the note |
Fallback chain. Inactive zone: use the instructor demo hostname for the edge and cache items, and say so in the note; the origin-direct measurement is unaffected. doctl blocked: use the Control Panel checklist and record the panel paths. Outbound probes blocked: use the bundled sample CSV and the published measurements. Cloudflare unavailable: measure the origin only and label the missing edge item. The evidence shape does not change; every substituted item is named.
Notebook
Open the lab notebook; it loads the bundled samples, accepts your measurements, asserts the shape, and writes latency_results.json. The reference run executes it headlessly on Python 3.14.2 with pandas 3.1.1, numpy 2.5.4 and matplotlib 3.11.2; it exits 0 and every assertion passes.