Once we'd established you did actually mean the box I don't think anyone really thought you were being daft anymore, but is a daft way of naming things.
Its interesting tbh, its only when I posted that I realized i had always called it the D when playing with friends. Just guessing but maybe its because as kids (or even now but more so back then), when you had to shout for a pen in hindi, you said its inside the D rather than the box, perhaps because its shorter and easier to say.