Stir together egg yolks, milk and brown sugar in a saucepan over medium heat. Cook 6-8 minutes, stirring constantly, until mixture reaches 170º F (do not bring to a boil).
Stir in peanut butter and ½-tablespoon vanilla. Continue to cook and stir just until peanut butter is well combined.
Remove from heat, transfer to a bowl, and let sit at room temperature for about 10 minutes. Cover and store in the fridge until completely chilled, about 1-2 hours.
In a large bowl, beat heavy whipping cream on high speed until soft peaks form, about 5 minutes.
Stir in powdered sugar and remaining vanilla, continue to beat until thick peaks form, 3-5 minutes. Carefully fold in cooled peanut butter mixture until combined.
Pour the ice cream mixture into a freezer-safe container. Swirl in or top with additional peanut butter or garnish with peanuts if desired. Cover and freeze at least 6 hours or overnight. Before serving, let sit at room temperature about 10 minutes to soften slightly.