As pointed in the comment below there are two =) I wish for Knuth to pop in here one day ;-)
Quality is attractive, make it public visible. If have a blog, write about excellent questions and answers on the site, quality is attractive. You could even post a story on hacker news, reddit, or stumbleupon. Share links to great questions.
Seeing an own difficult question answered here, is attractive. If you notice that a TeX hacker raises a question via twitter, facebook, mailing list, Usenet, or web forum and you see there's no good answer yet, post the question here. Help her or him to get the answer. Of course include a link to the source.
Make package authors interested by questions and answers which can improve the package or the class.
Directly invite TeX experts, for example as interviewee or guest author in our upcoming TeX.SX blog.
Go where the TeX experts are, show up at TeX conferences and local user group meetings, talk about TeX.SX.
I think building up the reputation of this site as a - nay, the - place to get high-quality answers to *TeX questions will be a major factor. Any experts who are interested in answering questions at all will probably want to be associated with the best site out there. And any experts who aren't interested in answering questions, well, we weren't going to get them anyway. Obviously, we're not going to spring out of private (or even public) beta with that reputation already built, but what we need to do now is make the decisions that will guide the development of the site in such a way that we're setting it up to be the best resource on its topic.
Besides that, it's probably mostly a matter of using our connections. As mentioned, there are a couple of real live LaTeX hackers registered and perhaps they know other people who would be interested in a site like this. Many of the rest of use are probably in academic environments, which means we're likely to have a lot of contact with other people who use *TeX and perhaps even have written packages for it. They'd be valuable resources for the site, if they're interested.
Improve the interface. I want to see a threaded interface in date order (latest last), with topics I have read or skipped greyed out. It needs to show only the OP of each thread by default, with the conventional [+] to expand and [-] to contract. In fact, I'd like to read the site like a Usenet newsgroup :-) but keep the vote-based ranking.