Short answer: script.py.mako
Longer story
So I wanted to fix the template I use for alembic revisions, and I always forget where to search for it.
The template is in script.py.mako file in your alembic directory.
For example, my alembic.ini is in db folder, and my script.py.mako file is located in db/migration.