Yeah, this part isnt well explained because they pull powers out of nowhere.
Sora uses the power of awakening to essentially "reverse" the moment that Kairi died to save her. This is the same power Sora used in the main game at the end to undo the final battle, preventing the cast from getting wiped out by the heartless and losing. However, its hinted by Chirithy (the cat thing from UX) that abusing this power will cause him Sora to disappear from existence. Sora does so anyway, travelling through gates to search for and eventually finding and saving Kairi from death.
This is why at the end, Kairi is alive but Sora "dies"/disappears. Where Sora goes is unknown but its implied in the DLC and now with the KH4 trailer that Sora has gone to Quadratum, the worlds Yozora comes from. We dont know that much about Yozora but it is also heavily implied that he has a similar backstory to Noctis from FF versus 13 (nevermind the theory of Luxord actually being the driver and former keyblade wielder).
We dont know much about Quadratum, the world Sora finds himself in the trailer, but we know that the girl he meets is Strelitzia, a former keyblade wielder from UX who was also killed so these are unknown threads.
Yeah, Pete and Malificient were basically red herrings, searching for "the box" which is the box Xigbar/Luxu was carrying at the end. They basically show that the box is important (likely because it was given to Luxu by the master of masters) but they are basically background, doing their own thing and not getting involved in the main story.
If I made you more confused by answering, Im sorry lol.