Managed objects are normally discovered by the garbage collector by walking AppDomain statics and the stacks of the running threads and the objects they reference on the GC heap. But there are certain scenarios where the collector itself is not capable of finding a reference to an object that's live and should not be collected.
This happens when unmanaged code uses such objects. This code is not jitted so the GC has no good way to discover the object references back, the stack frame of such code cannot reliably be inspected to find the pointer back. You must ensure that the GC still sees a reference. Which is what GCHandle does. It uses a dedicated table of GC handles built inside the CLR. You allocate an entry into this table with GCHandle.Alloc() and release it again later with GCHandle.Free(). The garbage collector simply adds the entries in this table to the graph of objects it discovered itself when it performs a collection.
The gcroot<> keyword in C++/CLI is an example of this. It allows writing unmanaged code that can store a simple raw pointer and have it converted back to the managed object reference where needed. The GCHandle.ToIntPtr() method generates that pointer, FromIntPtr() recovers the object reference. The GCHandle table entry ensures the object won't be collected until the unmanaged code explicitly calls Free(). Usually done by a C++ destructor.
GCHandle also supports the ability to pin an object, used when you need to marshal to native code and the pinvoke marshaller can't get the job done. You'd pass the return value of GCHandle.AddrOfPinnedObject(). And it implements weak references, although you'd normally always use the WeakReference class for that. And it implements async pinned handles, allowing pinned I/O buffers to be unpinned automatically on an I/O completion, a capability that's not directly exposed. So yes, GCHandle does more than just keeping a reference.
And significant to your question about delegates, the CLR supports using a delegate to call native code. The underlying helper function is Marshal.GetFunctionPointerForDelegate(). Which dynamically builds a machine code stub that allows native code to callback into managed code. This requires the delegate object to stay referenced, GCHandle.Alloc() is often used for that. It isn't the only way, storing the delegate into a static variable is another way to ensure that the delegate object won't be garbage collected. That delegate doesn't otherwise have to be pinned, a GCHandleType.Normal is fine.
This is used a lot in any .NET program, but most of it is out of sight. Particularly in the .NET framework code itself and in the pinvoke marshaller. Having to use GCHandle to protect a delegate object is only necessary when the native code stores the function pointer and can make callbacks later.
