Head down the tunnel underneath Norte Dame. Just a bit past where you encounter the first group of hostile monks you will find another pair. They stand guard in front of a gate that is locked. You can either lure the guards away with a Cherry Bomb lobbed further down the tunnel or by simply removing them.
Turn to the right-hand side of the doorway. You will see a switch. Interact with it to open the gate. Head inside and go over to the desk on the left. You will see the Elixir you are looking for there. Claim it. Be sure to also loot the chest that is around