It's important to know the difference between removing and deleting before you start to clean out your collections.
1. Remove from collection
If you want to remove clips from a collection, you have to click on the "unlink" sign in the toolbar above the grid. Your clip won't be in the collection anymore but will still be in the main library.
2. Delete a clip
To delete a clip permanently from your library, click on the trash can in the toolbar or choose "remove clip" in the clip menu. Your clip won't be in the main library any more.