The defences, which date from at least the 16th Century, were discovered by archaeologists during foundation works for new Military Tattoo stands.
Service trenches were opened which revealed two structures about 2m beneath the esplanade.
Archaeologists will record the remains before they are reburied.
The first trench, which was discovered earlier this week by experts from CFA Archaeology, revealed the remains of a wall about 2ft wide, which is thought to be part of the north perimeter boundary wall between the city and the castle.
The opening of a second trench uncovered what initially looked like a continuation of the boundary wall, but now appears to be separate remains, similar to the foundations of a spur, a 16th Century defensive bastion which protected the entrance to the castle.