This paper proposes an arithmetic of 3-D house rebuilding based on image understanding according to the standard rules of graphics in the architecture industry. The arithmetic picks up midlines of walls from the architecture drawing in order to obtain the whole topology of the house combining the rules of architecture drawing, image understanding and artificial intelligence; and then distinguishes two important members: door and window from the graphic; at last rebuilds the 3-D house. The result of the experiment indicates that the arithmetic is effective for actual architecture drawings with the help of man-machine conversation.