My brother and I experienced this with my PS4. We had the same issue. It's because of a dust pile up in your controller. I am afraid to open up my controller, so I went out and bought a new controller.
Also, make sure you update your console, and your games as they come.
I'm not saying updating would solve the problem, but I would advise it, to keep it at latest version, and see if its not the game or console. Ruling them out. For me, I figured it out that it wasn't the console or the game, it's the controller. Around the same time I found out that Nintendo Switch has a similar issue, called drifting.