The IND Eighth Avenue local train (C) and Sixth Avenue local train (B) stop at Central Park West and 72nd Street, the closest subway stop to the Naumburg Bandshell. The IRT Broadway line #'s. 1, 2, & 3 trains stop a few blocks west of the park on Broadway at 72nd Street. The Lexington Avenue local, #6 train, gives you a choice: Stop at either 77th Street or 68th Street. Please note that Broadway and Lexington Avenues are only a somewhat short walk away from the Naumburg Bandshell's mid-park 72nd Street location. The Naumburg Bandshell is just south of the Bethesda Fountain and Terrace.
Central Park is bordered on the west by Central Park West and on the east by Fifth Avenue. South of the park CPW is Eighth Avenue. North of the park (Harlem) it becomes Frederick Douglass Blvd. Fifth Avenue's name remains the same throughout its length.
There are also buses running on Central Park West and Fifth Avenue (downtown) and Madison Avenue (uptown). There is a bus map at this site (you'll need Adobe Acrobat) http://www.mta.nyc.ny.us/nyct/maps/manbus.pdf
If you are traveling a short distance, take a bus, but otherwise, they are slow. Or take both the subway and the bus, if you need to. A MetroCard gives you a free transfer.
Taxis can drop you at 72nd Street, on either the east or west sides. The east side approach offers the opportunity of either going into the Park and trying to be let down at the Concert Ground, or just along the East Drive within the Park at the 72nd Street transverse. One should also be aware that the east side is marginally closer to the bandshell.
Driving and parking spaces outside the park are scarce and costly.
There are also bicycle rickshaws available around the Park, they can bring you quite close to the Concert Ground.
Or, perhaps, another very convenient option: your own bicycle.
In any case, please try and attend the concerts, they are well worth the trip. The music and setting are wonderful.