On the outside (which can be seen as part of the main backgrounds in both games) the castle itself has a very cliché appearance akin to villainous bad guy castles. Having darkly colored bricks, several stories high of fortress walls and pointy-roofed towers with round-door shaped windows that glow yellowish light colors in both day and night, it also has a very large gate with a frightening grim face. When the player reaches Count Batula's Mansion in the Spooky chapter; they will be able to see the castle in a sideways view. It should also be noted that it can also be seen at the entrance to the Feral Reserve Bank, as it is part of the castle.
It is unknown if the castle originally belonged to Professor von Kripplespac during his days as the Weasel King before he lost his legs in the Milk Wars, but the connection is possible. the obvious hints that Kripplespac WAS the owner of the castle are the Milk Wars, the Weasel guards in the castle and the bank, and his laboratory in which several of his Tediz experiments take place in.
It's only right after Conker defeats Heinrich in the Throne Room at the end of the game that the whole castle belongs to Conker when he's declared king of all the land.