Between reading in my kindle app and listening via audible I've already flown through Chamber of Secrets and Prisoner of Azkaban. Neither is quite as good as I remember from when I read them as a kid, but both hold up pretty well.
If memory serves, then the series really picks up with Goblet of Fire and stays interesting all the way through the end. I'll soon find out since the next book I have queued up is that one!
It's been really interesting revisiting Harry Potter as an adult. I first read these books as a teenager and enjoyed them a lot. As an adult I can find more plot holes in them and pick them apart more critically but I'm still able to have fun with them and enjoy them for what they are.