Harvey Barnes handed Newcastle a sixth minute before Pierre-Emerick Aubameyang struck twice to earn OM a crucial three points.
Afterwards, Howe refused to blame goalkeeper Nick Pope for Aubameyang's first, though was left disappointed with the overall defending from his team.
Howe said: "I think it's a concern when we concede goals, regardless of who's at fault. I think we have to look at everything with a balanced head.
"He saved us many times. He made some really good saves against Manchester City just two days ago. That's the life of a goalkeeper. But I'll certainly back him."
He also admitted: "We got what we deserved. It was not a West Ham or Brentford.
"There are areas we can improve. These are the games we want to play in and test ourselves in.
"The second half I think we did not react to the goal. We did not react to the goal, we were on the back foot and the crowd came into it. Then we were not at our best.
"I will learn more from watching it back. We need the intent to go for more goals in games. The intent was there. Maybe our press was not as intense as it has been.
"1-0 is a delicate scoreline and we did not defend well enough. The result leaves us bruised. We have great games ahead."