After you rescue Vane, keep after the Pearl. Now is when you can start firing on it to start disabling it. Charge in and ram it to get some good damage in (especially if you are after the optional objectives). There are a few schooners and a brig that are launched from the nearby fort. Take them out first to make your life easier. This is the time to take care of the second optional objective by weakening the other ships and letting the fort take out one.
When it is time to deal with the Pearl, you need to kill 15 crew and 2 Scouts. Start by using the Swivel Gun to go for the powder to reduce the crew drastically. The scouts are found in the rigging above you. This is why it is important to go for them second as it takes a lot of time (or diving into the water and climb) to rejoin the fight with the crew. Just get on through this boarding action.