The cube, or hexahedron, has six square faces, eight vertices and twelve edges, with three faces meeting at each vertex. Euler's formula holds as it must for a convex polyhedron: 8 - 12 + 6 = 2. The angle between two adjacent faces is exactly 90 degrees, and the cube is the only one of the five for which that angle is a whole number of degrees.
It is the dual of the octahedron, and the pairing is exact rather than approximate. The cube has six faces and eight vertices; the octahedron has eight faces and six vertices. Mark the centre of each square face of a cube, join the neighbours, and an octahedron appears inside it. Do the same to the octahedron's eight triangles and a cube comes back. The edge count of twelve is unchanged in both directions, and the two solids share one symmetry group of 48 operations, 24 of them rotations. Any statement about the symmetry of a cube is a statement about the symmetry of an octahedron.
The cube's other distinction is practical rather than ornamental. It is the only regular solid that tiles three-dimensional space: identical cubes stacked face to face fill a volume with no gaps and no overlaps, and none of the other four can do it. Aristotle believed regular tetrahedra packed perfectly as well, and he was wrong; the mistake was repeated for a very long time before it was properly corrected. That single tiling property is why the cube, rather than the more elegant icosahedron, became the shape of the built world. A good deal of what gets written about the cube as a symbol of order and foundation is downstream of a fact about packing.
One caution about that symbolism. The cube is stable in the sense that it sits flat and stacks, but it is not rigid. A cube built from rods and hinges racks over into a leaning box, while a tetrahedron built the same way cannot deform at all. Stability and rigidity are different properties, and the cube has only the first of them.