I'd be careful about that. Something smaller may work for a while, but in the end it's going to end up having the same growing pains as SO/SE...
You start with a group of likeminded people, and if that group does stick together and no one leaves you might not need new people... but when new people arrive that aren't as likeminded, you'll end up with conflict.
You start with a group that has a certain knowledge level about stuff. As the group sticks together and learns, it becomes more expert-y. What happens when new people arrive, that ask questions that the 'experts' deem to easy?