There are some aspects of graduate school that are more daunting than others, and finding a research topic is perhaps the biggest obstacle for most students. The characteristics of an ideal topic are to some extent incompatible:
On the other hand, if a field is too crowded, and the subject too prominent, then you risk being ``scooped" by a more experienced researcher who is able to work faster than you. In this case, you may be forced to start over again (rather disastrous) or at least publish jointly (possibly a blessing, but surely an inconvenience).
On the other hand, it is impossible to work in a vacuum, and your task can be significantly harder if you don't have a group of people working on closely related problems with whom you can interact and share code.
On the other hand, a multiyear plan of research is a very valuable asset.
On the other hand, an ideal subject is of no use without a thesis advisor who is willing to direct you in it.
Clearly some compromise is necessary here!
It is very important to make the transition from the passive mode of learning that traditional lecture courses encourage to an active and critical learning style. Whenever you read technical material, evaluate a piece of software, or listen to a research talk, ask yourself these canonical questions:
One technique that some find helpful is to keep a written log of technical reading and listening. Review it periodically to see if some of the ideas begin to fit together.
Set aside some time every week for trying to generate research ideas. Some possible catalysts are:
Add these to your log, and ask the canonical questions. As you review the log 6 months from now, you may find something that has become important to you but was beyond you when you first encountered it.
Which comes first: the thesis advisor or the thesis topic? The answer is, both ways work. If you have identified a compatible advisor, you could ask for an independent study course. Both of you together set the focus for the course, with you having more or less input depending upon your progress in identifying a subfield of research.
Once you have identified a topic that looks feasible, make sure you are aware of all of the literature in the area. Keep reading and listening, and keep distinct in your mind what is different between your work and others. If you do not frequently review the literature you read months ago, you may find yourself unconsciously claiming credit for other people's ideas. On the other hand, don't let other people's frame of mind limit your creativity.
It is possible to spend almost all of your time in literature review and seminars. It is easy to convince yourself that by doing this you are working hard and accomplishing something. The truth of the matter is that nothing will come of it unless you are an active reader and listener and unless you assign yourself time to develop your own ideas, too. It is impossible to ``finish a literature review and then start research." New literature is always appearing, and as your depth and breadth increases, you will continually see new connections and related areas that must be studied. Active listening and reading must be viewed as ``continuing education'' that will involve you for the rest of your career. Don't fool yourself into thinking it must be finished before you can begin research.
From reading, interacting with your advisor during independent study, or working on a research assistantship, some possible projects will emerge. Make a list of open problems and possible projects that are of interest to you, and discuss them with potential advisors.
Even after you have decided on your initial focus, it is important to continue a routine of reading new material and attending seminars. All of these sources can contribute to the development of your idea.
At this stage you can add one question to the canonical list: How can these ideas help me solve my research problem?
Remember that often the initial idea is quite far from the final thesis topic. If you remain active in reading and listening, it will be much easier to generate alternative topics if the time comes.