Normally a vacuum is only needed when you are doing bulk deletes from the table (it essentially reclaims dead space) not after running inserts.

The analyze part of the vacuum could be the reason for this. If the table was empty before then PG might choose the wrong execution plan (but then 50000 rows is a very small table anyway).

It would have been interesting to see the execution plan (by running EXPLAIN) right after the insert