When driving that way, the first place to see is the city pillar shrine (Lak Mueang), a small shrine located in a nice park with the wooden pillar covered by gold in the center. And as it has a pond within the park, one can of course by food to feed the fish inside. And also as usual with such attractions, it has several souvenir carts as well as small eateries at the parking lot.
The actual offices are located 500 meter further down the street. Starting on the northern side, it has the district office (ที่ว่าการอำเภอ) of Mueang Phetchaburi district first. As there are a few additional buildings between the office and the street I could not get the standard view of the whole office this time - and now I look at the building in Google Earth again I think I have been on the backside only and thus missed the better view on the other side - so I have to return there to check next time.
A few buildings further down the street comes the municipality office (สำนักงานเทศบาล) of the town (thesaban mueang) Phetchaburi (เทศบาลเมืองเพชรบุรี).
Now turning south to the other side of the road it has the province hall (ศาลากลางจังหวัด), the central administration for the whole province. I am not sure what were the two smaller buildings in between - maybe the PAO office is among them-, but then on the other end of the block opposite the district office it has the provincial court (ศาลจังหวัด).
View Phetchaburi administration in a larger map