Certain documents are cheap and quick to get at home and expensive or impossible to get from Germany. Pack these properly:

Must-haves:

Passport (obviously) and several biometric photos (you'll need them repeatedly)
Birth certificate — with apostille and certified translation if not in English/German
All degree and school certificates + certified translations
Marriage certificate (if applicable) with apostille — essential if your spouse joins later
International vaccination record
Driving licence (an international driving permit helps for the first months)

Smart extras:

A folder of passport copies and spare photos
Any medical records / prescriptions (with generic drug names)
Police clearance certificate — some processes ask for it and it's much easier to order at home
A signed Vollmacht (power of attorney) left with a trusted person at home, so someone can request documents or act for you after you've flown

The German folder ritual: buy a proper A4 folder on arrival and keep every official paper. Germany runs on paper trails — the letter you almost threw away is always the one the Amt asks for.

Red flags:

Uncertified translations. German authorities generally accept only sworn/certified translations — the cheap online translation gets rejected at the counter.
Apostilles can only be issued in the country that issued the document. You cannot get your birth certificate apostilled from Germany — do it before you fly.
Leaving originals at home with nobody authorised to handle them (that's what the Vollmacht is for).
Exactly one copy of anything. Scan every document to cloud storage before packing — a lost bag should be an annoyance, not a catastrophe.