Someone asked this question before and I just found a way to fix it while writing start time into the Video ID.
Instead of adding &t=2000s or ?t=2000 after Video ID, it should be ?start=2000&
For example: I want to start this video (1k4x9FrD5k4) start at 30s, I change the video ID to 1k4x9FrD5k4?start=30&
Hope it helps.