Kaczynski made the statement on local radio on Monday as opposition parties seek to oust Justice Minister Zbigniew Ziobro, the leader of Solidary Poland, a Eurosceptic ally of PiS, in a no-confidence vote.
Polish politics has been rife with rumours of a split between Ziobro and Mateusz Morawiecki, the prime minister, over European policy, with the justice minister adopting a more confrontational stance towards Brussels.
"Prime Minister Morawiecki will certainly defend the minister in parliament and will vote together with his closest associates to reject the no-confidence motion," Kaczynski said, adding that the vote "certainly won't be a test of strength between Prime Minister Morawiecki and Minister Ziobro."
However, "there is certain tension" between Morawiecki and Ziobro, the ruling party leader admitted.
"The minister simply doesn't see different complications... and doesn't realise the scope of implications that Poland is involved in and must be involved in, as we're part of the global market and at the same time we're implicated in our membership in the European Union while our society, for a good reason, wants Poland in the European Union," Kaczynski said.
"When all of this is taken into account then, of course, there have been mistakes, but in general there is no alternative to the policy pursued by the prime minister," the PiS leader continued.
Later on Monday, the prime minister said he would support the justice minister in the no-confidence vote.
"Of course, I will defend the minister, of course, I will defend the unity of the United Right (the ruling coaliton - PAP)," Morawiecki said.
"I'm convinced that our government will be united despite the fact that our discussions are sometimes very open," he added.
Poland has been embroiled in a protracted conflict with the EU over the PiS government's changes to the justice system that Brussels says violate EU principles. Ziobro has criticised Morawiecki for being too lenient in the conflict as he accused the EU of "colonising" Poland. (PAP)