Google Page speed insights is not the correct way actually. It shows you a relative score depending on some factors instead of showing you the actual time your site is taking to load.
That actual loading time is what matters from SEO and UX perspectives.
I checked with GTmetrix (GTMetrix and Pingdom are two better alternatives to GPS Insights) and found that your website is really fast (Ref:- https://gtmetrix.com/reports/tricksinside.in/zVK2nS7M)
It loads in just 3 seconds when tested from Mumbai servers and that is a good sign. So for page speed, there is nothing to worry from your end.
BTW, As per as my knowledge, Analytics is showing you loading time more than 3-4 seconds just because, you have some ads out there at your site and they take bit time to load. And analytics give results depending on how much time all the content at the page takes to load .
Conclusion: - There is nothing to worry from your end. Still some things you can do from your end are:-
Host Analytics locally using CAOS plugin. This way you can solve leverage caching issue for Google analytics
Minify the possible CSS and JS (Ref: GTmetrix report)
Brag about your site’s good loading time among your fellows