SimpleGraph was originally designed for only simple graph functionality. But, later it improved to this level. This has caused lot of inconsistency in the code including dragging graph objects.
The only way to fix this issues is redesigning the component, sorry.