In a small pot or microwave-safe cup, bring ¼ cup of the water to a boil on the stove or in the microwave. Add tea bags and steep for 5 minutes. Remove and discard tea bags.
Stir in sugar until completely dissolved and add remaining ¼ cup of cold or room-temperature water. Set aside.
Fill 1 16-ounce cup with 1½ cup of ice. Add milk.
Pour chai mixture over the milk and stir to combine. Top with optional whipped cream if desired, and enjoy!
Notes
* If you typically prefer your drinks less sweet, start with 1 teaspoon sugar and add more to taste.