File Type Identifier: Technical Postmortem | Security Writeup by Husain Hakim
A deep dive into why relying on file extensions is a security flaw, and how I built a custom magic number identifier from scratch.
File Type Identifier: Technical Postmortem
A deep dive into why relying on file extensions is a security flaw, and how I built a custom magic number identifier from scratch.
## The Problem: Extension Spoofing
In the real world, file extensions are essentially meaningless labels. Operating systems like Windows heavily rely on them to determine how a file should be opened, which creates a massive blind spot. Threat actors frequently exploit this by taking a malicious executable (`payload.exe`), renaming it to something benign (`invoice.pdf`), and deploying it via phishing campaigns.
To the untrained eye (and unfortunately, to many naive email filters and endpoint security tools), it looks like a harmless document. When the victim double-clicks it, Windows simply executes the malware. To counter this, defenders cannot trust the filename; they must look at the actual bytes.
## Why Magic Numbers, Not Extensions
The starting assumption behind this project was simple: extensions are metadata, not truth. I wanted to build something that reads the actual bytes instead of trusting the filename, the same way `file` on Linux does, but with a forensics-lab presentation layered on top so the tool felt purpose-built rather than like a script with a UI bolted on.
## The Signature Database Tradeoff
The first real design decision was how to store and match file signatures. I could have pulled in an existing Python library that already maps magic numbers to file types, as there are a few solid ones. I chose to hand-build a signature database of 30+ common formats instead, mapping raw hex sequences to file types myself.
That was slower to build and definitely less comprehensive than a mature library would be (I'm covering the common cases like PDF, PNG, ZIP, ELF, MZ, but not the long tail). But it meant I actually understood every signature I was matching against, instead of treating file detection as a black box I called into. For a security tool specifically, I wanted to know exactly what triggered a "spoofed" flag rather than trusting a dependency's internal logic I hadn't read.
## Where Binary Handling Got Messy
Reading files in binary mode sounds trivial until you're building the hex dump view. Formatting offsets, aligning hex pairs, and rendering the ASCII sidebar (with non-printable bytes masked) took more iteration than I expected, mostly around edge cases like files shorter than the byte window I was reading, and getting the offset formatting to actually line up visually instead of drifting after a few rows.
I also made a deliberate choice to analyze files entirely in memory rather than writing them to disk first. That was partly a performance decision and partly a security one: a forensics tool that writes untrusted uploads to disk before inspecting them is introducing exactly the kind of risk it's supposed to help catch. In hindsight, that constraint shaped a lot of the backend logic more than I initially planned for.
## What I'd Do Differently
Polyglot detection is thinner than I'd like. Right now the tool can flag a basic extension mismatch, but true polyglot files (valid in multiple formats simultaneously (a ZIP appended to an executable, for example)) need more than a single signature check at offset zero. I understand the concept better now than when I started, but the current implementation only scratches the surface of actually detecting them reliably.
Test coverage was an afterthought. I built and validated this mostly by hand, dragging in real files and checking the output looked right. That worked for a solo project on a deadline, but it's not how I'd want to ship something people might actually rely on for a real spoofing check. A proper test suite with known-good and deliberately malformed sample files is the obvious next step.
The UI came before the edge cases. I'm genuinely happy with how the forensics-lab aesthetic turned out: it does what I wanted, which was make the tool feel like something built for the job rather than a generic upload form. But I spent real time on that visual layer before hardening the backend against malformed or truncated files, and a couple of early bugs (crashes on files under my byte-read threshold) were a direct result of that ordering. Next time, correctness first, presentation second.
## The Actual Takeaway
This project taught me more about the gap between "detecting something" and "detecting something reliably" than I expected going in. Matching a magic number is easy. Matching it correctly, handling the edge cases, and being honest about what the tool can't catch yet (polyglots, more exotic formats, deliberately crafted evasions) is the harder and more interesting problem, and it's the part I want to keep pushing on.