First, be sure the game isn't running on integrated graphics. Then I'd check and be sure that your GPU isn't thermal throttling. Open a temperature monitor such as MSI Afterburner to check the temperatures of your GPU. You can also try running 3DMark to benchmark your system and try to identify the cause of your FPS drops.