Heat the oil in a large pot over medium heat. Add the garlic, lemongrass, and red pepper flakes and cook until fragrant. Add the vegetable broth, fish sauce, lime juice and brown sugar. Bring to a boil, then reduce heat and simmer for 15 minutes.
Strain the soup into a clean pot or bowl. Serve hot with your favorite toppings.