For the Chocolate Cupcakes: Preheat oven to 350 degrees F. Line 2 cups of a 12-cup muffin tin with paper liners. Spray with nonstick cooking spray.
In a medium bowl, whisk together flour, sugar, cocoa powder, baking soda and salt. Whisk in milk, oil and vanilla until just combined. Divide batter evenly between cups.
Bake for 20-22 minutes, or until a toothpick inserted in the center comes out clean. Let cool completely before frosting.
For the Nutella Frosting: With an electric mixer in a medium bowl, beat butter and Nutella until smooth. Mix in powdered sugar until well blended. Add vanilla and 1/2 teaspoon milk; mix until incorporated. Add additional milk if necessary to achieve desired consistency.