Any second hand store will have racks of sweaters, but they are not all appropriate for frogging. I like to check the tags for fiber content. I generally don't buy any acrylic sweaters, since acrylic yarn is cheap enough to buy new and I rarely knit with it anyway. I try to find knit items that are mostly comprised of natural fibers (wool, cotton, silk) -- they are often blended with manmade materials (rayon, nylon) for durability, which I don't mind. I think of this as an opportunity to find yarns that I couldn't afford otherwise, so I try to stay disciplined when shopping. Also, it is important that the knit fabric has not been felted; make sure that the individual stitches are visible. The most important thing to look for in an item for unraveling is the type of seams the garment has. If there is zig-zag stitching or serging covering the edges of the seams, this is NOT a good sweater for unraveling (see top photo). It means the pieces of the garment were cut from a larger piece of knit fabric and the edges were stitched over to keep them from unraveling. This type of garment would yield only short lengths of yarn, rather than one continuous piece. Instead, choose garments that have slightly rolled, bound edges; this means the pieces were knit individually to size, then seamed together (see bottom photo). Often, even on the "good" garments, collars or cuffs will be sewn on with zig-zag stitching or serging, but as long as the main body of the garment is comprised of individually knit pieces it should yield useable yarn.
To begin, I studied the way the garment was put together. From a knit skirt, I removed the waistband (if it was a sweater, I would remove the collar and sleeves). Good eyesight helps here, to make sure that you are cutting the thread that grafted to the garment pieces together, not the threads in the pieces themselves. Along the side seam, I pulled the edges of the front and back sides of the skirt apart, revealing stitches between them that looked like the rungs of a ladder. Using cuticle scissors, I snipped the first "rung" of this thread. This is the thread that is holding the garment together along the sides. I noticed that if you start cutting the side seam from the top of the skirt (or at the armpit in a sweater), you can sometimes pull that laddered thread and it will easily rip out all along the seam. In other areas, I just used the cuticle scissors to cut the thread if it did not pull out easily. Once I had separated the front and back sides of the skirt, I started at the top and pulled on the uppermost horizontal running thread. Several short pieces may come off, but eventually the thread will be one long continuous piece. As I ripped, I wound the yarn into a ball. Once the ball weighed 50 g, I would cut the yarn and start a new ball. The skirt yielded a little over 200 g, 4 balls.