This is a three part story. The memories are mostly canon compliant, but I added a bit to Pitch's speech. You know, for the angst.
Trust is fragile. This is not an opinion so much as a fact, a commonly held belief among the majority of people. Typically, trust is not given much thought until it is questioned. This is because trust, like many connections, is built intuitively and subconsciously- a tenuous bond that grows stronger over the course of a relationship without conscious effort needed from either party. As long as the relationship is strong, the trust will develop on its own, naturally.
Unfortunately, trust is less tenacious than most other aspects of a relationship. Anger can be forgotten; wrongs can be forgiven; but trust is nearly impossible to rebuild.
Luckily, there was no trust between Jack and the Guardians. After all, trust cannot be formed without the foundation of a relationship. And aside from a few incidents where Jack had crossed paths with the Guardians – the most recent being the infamous Easter blizzard of '68 – there was nothing to suggest that they even knew each other, much less that there were any sorts of connections involved. Sure, Jack and the Guardians knew of each other, but that was mostly though stories and legends, of which the most important information was left out and the less pertinent exaggerated. No, there were no true connections, no actual bonds upon which trust might have been built.
By the time the Guardians decided to get to know Jack, approximately 300 years after Jack's 'birth,' it was too late, at least immediately. Maybe 'too late' is the wrong phrase, it was just… really bad timing? The problems were too large, the danger too imminent, for time to build such frivolous things as relationships and trust. Perhaps those could come later, after Pitch was dealt with, but during the first proper introductions nobody had the luxury of time; the Guardians had to trust that Jack, while chaotic and seemingly amoral, would fight with them, for his memories if nothing else. Jack had to trust that the Guardians would have to have his back during any fights, because surely their indifference to his existence would have to fade, however briefly, if he was to find himself in immediate danger of their sworn enemy?
As it would happen, during their adventures fighting Pitch and helping Tooth collect teeth, everyone did grow closer. It was an unorthodox arrangement to be sure, and none of the relationships formed could necessarily be called 'strong,' but it was a huge improvement from the previous few centuries.
After Pitch was defeated, there was an unconscious but major shift in Jack's relationship with his fellow Guardians. Instead of being united against a common enemy, their relationships were now built on Jack's newly formed status as a guardian and the bonding time implied in the formation of that status. Everyone had generally returned to their normal routines, with the exception that all of the Guardians decided to remain closer; both to each other and to the children they were meant to protect. This provided ample opportunity for Jack and the other Guardians to get to know each other better, and for real relationships to be formed.
Unfortunately, building relationships is inherently hard; compounded with their less than stellar history, it would take a long time for Jack and the Guardians to truly trust each other.
Oi, mate, it's only been three weeks!" Bunnymund said as he hopped out of a rabbit hole in the middle of North's workshop. "What else could have possibly happened for you to use the Borealis again already?"
"Yeah North, can we make this quick? The sun's setting in the western hemisphere and you know how busy it gets in the summer." Toothiana didn't wait for a reply as she turned around and started giving orders to the baby teeth buzzing around her.
Sanderson used his dream sand to create a question mark above his head, a confused and mildly concerned look on his face.
"It is about Jack," North said in his booming, attention-grabbing voice.
"Jack?" Tooth looked over in surprise, momentarily forgetting her job.
"Where is that little bugger, anyway?" Bunny asked irritably. "He knows he's supposed to be here. If that snowflake thinks he can slack off on his responsibilities already, he's got another thing coming. Well I'll-"
"Bunny," North said sharply. "Enough. Jack is currently sleeping in a spare room. I wanted to use this time to talk to all of you, while Jack is now here."
"But why? Did something happen? Is something wrong?" Tooth shooed off the last few of her fairies, trusting them to remember the schedule. She had been working on that recently, passing along some of the responsibility so that she would have a chance to live in the present.
"Nothing happened. And I don't think that anything is wrong, really, it's more that…" North paused, looking up at the moon in the sky, as if contemplating his next words carefully. "Have any of you noticed anything… odd about Jack? Or that he doesn't act the way that you would expect him too?"
Sandy looked more confused, the sand above him morphing from a snowflake, to a question mark, to a dolphin, and back to a snowflake.
"Odd? What are you talking about, North?" Tooth looked between North and MiM anxiously, as if the moonlight to hold all the answers.
"Yeah, mate, you're not making a good lot of sense right now," Bunny for his part was at least making a conscious effort to understand what North was saying, instead of writing off anything having to do with Jack as 'mischief' like he normally did. "Frostbite's been irritating as usual for me."
"I am not talking in terms of his attitude," North explained, still struggling with what he meant to say. "I meant as if Jack sometimes acts… distant. Or as if he does not feel as comfortable as he should around us."
Tooth looked surprised. "Well yeah, he acts distant sometimes, but that's probably just because he still barely knows us. He's only been a Guardian for three weeks, after all."
"I know," North said quickly. "It is just that I expected him to get used to us a bit faster than this."
"I know what you mean, mate." Bunny, surprisingly, was the next to speak. "He still acts like an outsider. Like he's not part of the group."
"Yes, exactly!" North exclaimed. "That is it. Jack acts apart from our group."
Sandy got their attention again, forming images of Jack and the rest of the Guardians, with Jack separated from the group. A clock spun between them.
"Sandy's right," Tooth said sadly. "Three hundred years alone is a lot to get over. He's probably just still getting used to all the sudden changes."
"But I still feel as if something's wrong," North insisted. "He acts different around us than he does the children, and even the Baby Tooth. I feel as if he still does not trust us."
Bunny looked a bit uncomfortable, while Tooth blushed slightly. "I guess that probably makes sense. We haven't really given him a reason to trust us, have we?" she said.
Sandy was annoyed. This conversation was reaching the point where Sandy always got left out. After the battle with Pitch, Sandy had been hard-pressed to find out what happened while he was gone. It was as if nobody wanted to talk about, like it was something better forgotten despite the fact that it obviously still mattered.
It was at this point that Sandy brought up the same images that had ended the conversation in the past; a cloud of sand growing smaller and disappearing; the other guardians, together under a question mark; Pitch appearing next to them as a particle of sand grew into a cloud, reforming Sandy.
Tooth looked uncomfortable; Bunny, annoyed. North, for his part, did not immediately brush off the subject like he normally did and instead looked at MiM again thoughtfully.
"Perhaps you are correct, Sandy," North said, stroking his beard. "Maybe it is time we re-examine those moments. They might help us understand more about Jack. But I am afraid we can only tell you what we know."
Tooth was conflicted; she had been debating whether to view Jack's memories from that time by herself, since she knew Jack would never talk about it. She always managed to talk herself out of it with the reminder of how wrong it was; she was entrusted to protect memories from prying eyes, and that included her own. But now the Guardians were planning to relive the event anyways, and Tooth knew that Jack's memories could provide useful insight on whatever Jack was feeling.
"Actually, I think... I have a method that will let us hear it from Jack's side," Tooth said hesitatingly.
"You're not talking about those teeth, are you? You know those only hold childhood memories," Bunny said.
"Yes, Bunny, I am quite aware of how my teeth work, thank you very much!" Tooth snapped. "No, what I meant is... well, I think I'd better show you. It's over at Tooth Palace."
"Well, what are we waiting for then!? Let's go!" North said enthusiastically, pulling out one of his globes. "To Tooth Palace!" he shouted, throwing the globe on the ground. A portal appeared in front of them.
The group soon found themselves standing inside a small but intricate room inside Tooth Palace. Tooth had led them through several narrow passageways and a few locked doors before they finally arrived here, in a room filled with pedestals on which stood a wide variety of objects.
"This is the immortals room," Tooth said as she led the way to the center of the circle of pedestals. "It contains important memories from the lives of all the immortals."
"What!? You have all our memories locked up in some room!? I thought you only dealt with kids' memories!" Bunny was shocked and slightly horrified. North and Sandy were completely surprised too; in all their years, they had never even heard of a room like this. With so many important memories, it was amazing that Tooth had managed to keep it secret for so long.
"I deal primarily with childhood memories," Tooth replied, not looking very comfortable with the way the other Guardians stared around the room. "But I also protect the memories of immortals. With such long lives, sometimes memories need to be restored. And sometimes giving an immortal one of their ancient memories will help spark a solution to a current problem. They're sort of like history books; they're meant for knowledge more than nostalgia. I have to keep them protected like the children's teeth, but I also have to keep them hidden. These memories can be very dangerous if they get into the wrong hands. That's why I need you guys to swear to never tell anyone."
Sandy, the first to get over his shock, nodded respectfully. Eventually North and Bunny also managed to tear their attentions away from the pedestals to agree with Tooth's rule.
"So uh, how exactly do these things work?" Bunny asked, stepping closer to examine an object on one of the pedestals.
"Each immortal has an object relating to their powers or their personality. That object is what holds the memories," Tooth explained, flying over to a pedestal on the right side of the room. "See Bunny, this one's yours."
Bunny and the others surrounded the pedestal, staring down in wonder at the basket full of intricately decorated eggs on it.
"Each egg holds an important memory from your life," Tooth said as she turned away. "But we don't have time for everyone to see their own memories, we really shouldn't even be in here right now. Now let's see, Jack Frost... oh, yeah, here it is!"
Tooth floated to a pedestal near the front of the room, and off to the left. On it were tons of individual snowflakes floating restlessly across the pedestal, as if being moved around by a wind.
North and Sandy followed her, with Bunny trailing behind. Bunny kept glancing back at the egg basket, battling the curiosity of what his most important memories could be.
Before anyone could say anything, Tooth was speaking again as she reached out towards the snowflakes. "You can tell by the design of the snowflake which memory it holds. The one we're looking for should be right... here!" Tooth exclaimed as she caught one of the snowflakes in her hand. She pulled it off of the pedestal, giving everyone just enough time to see it glow blue before the memory appeared in their heads.