Unfortunately, the PIN Google Home / Google Assistant requests to open or close your garage door  or gate can not be disabled or avoided. This is a safety measure required by Google itself to prevent the operation of your Garage door or Gate from a stranger.

Google Assistant garage door opener - Can I avoid or disable the PIN Google Home / Google Assistant