Have you restarted your device since removing the app? If you've simply deleted the .apk using Root Explorer then then it won't have performed the 'cleanup' that normally happens when uninstalling an application. Restarting your device should cause the app drawer to re-detect the installed apps and the icon should disappear.
This answer (relevant part quoted below) explains a bit about what happens when uninstalling an app 'properly', this won't have happened if you've simply removed the apk file.
When you uninstall a package, it also removes other data to do with that package: for example, the app's own private data, its data on the SD card, your default preferences for that app. It also tells any other interested apps that you've removed the app, via an intent broadcast. Other apps receiving that broadcast might take further action based on that: for example, a launcher (home screen app) would remove desktop shortcuts and widgets from that app, since they won't work any more.
Also, make sure you have checked /data/app as there may be another apk for Google Drive there if you have updated the app via the Play Store. If you remove the apk from here too (if it exists) then the icon should disappear. A reboot may still be required.