Preheat oven to 400°F (200°C). Line a baking sheet with parchment paper.
In a medium saucepan, combine water, butter, and salt. Bring to a boil over medium heat.
Remove from heat and quickly stir in the flour until a dough forms and pulls away from the sides of the pan.
Return the pan to low heat and cook, stirring constantly, for 1-2 minutes to dry out the dough.
Transfer the dough to a bowl of a stand mixer fitted with the paddle attachment (or use a hand mixer). Let cool for 5 minutes.
Add eggs one at a time, beating well after each addition until the dough is smooth and glossy. The dough should be thick enough to hold its shape.
Drop rounded tablespoons of dough onto the prepared baking sheet, about 2 inches apart.
Bake for 15 minutes at 400°F (200°C). Then, reduce oven temperature to 350°F (175°C) and bake for another 15-20 minutes, or until golden brown and puffed.
Remove from oven and prick each puff with a small knife to release steam. Cool completely on a wire rack.
Prepare filling: In a saucepan, whisk together sugar, cornstarch, and egg yolks. Gradually whisk in milk. Cook over medium heat, stirring constantly, until thickened. Remove from heat, stir in vanilla extract. Cool completely.
Once cooled, fill the puffs with pastry cream using a piping bag or a spoon.
Notes
Dust with powdered sugar or drizzle with chocolate glaze before serving for extra indulgence.