You can create star polygons in 2 dimensions by extending the edges of a polygon until they meet. In a similar way, you can create star polyhedra by extending the faces of a polyhedron until they meet at an edge, forming the edges of the star polyhedron. The faces will be intersecting polygons that lie in the same plane as the faces of the original polyhedron. This process of extending the faces is called stellation.
You can not stellate the cube or tetrahedron and create any new shapes. If you stellate the octahedron you only create one new shape, the stella octangula, which is like two intersecting tetrahedra. However, when you stellate the dodecahedron and icosahedron, some exciting things happen.
If you want more information on how to make these models, the classic book is Polyhedron Models by Marcus Wenninger.
I drew this picture to show the 3 stellations of the dodecahedron. The top row shows the 'stellation pattern', which determines the shapes of the faces. I have tried to show how the faces intersect.
Me and my students have made various versions of these stellations using different materials.
The stellation pattern of the icosahedron on the right gives its many stellations. There are 59 in all, which you can look at here before you make them all!
The intersecting lines are created by marking 2 points along each edge of the large equilateral triangle in the ratio 1:phi. Note that when you do this, the ratio of the small length (between the two points) to the larger ones (point to corner) is also 1:phi.
I made one of these stellations which turns out to be a compound of five tetrahedra! The stellation pattern for this is shown in the slide show below, along with tips for construction. (click to play)