That was a building in the Brick Yard & think it had a name like the 'Puddling House or building' where clay was worked on and mixed with water & formed into bricks before going into the Kilns.
Here is a 1895 map.
In WW2 the Royal Navy used it for storing Shells and bullets for their guns.