Absolutely. My favorite breed rescue is Siamese Cat Rescue Center, and my neurotic meezer Beau came from there; in fact, this past February was his 4th Gotcha Day! I was so grateful to them for letting me adopt my Beaubert that I made purrpads for them for about a year.
I would so do it again. I'd probably take on an older cat, or ones with special needs -- Beau was a special needs cat, he was really traumatized by his earlier experiences, and it's taken every minute of the past 4 years to get him to be the relatively sane, sociable cat he is now!
ETA: would you post a link to your favorite rescue? You might help bring a person and a pet together!