So the long-short of his narrative: Joshua had a twin brother, Merkel. He is an orphan after his parents and brother died in a house fire. However, in the last seconds of his brother dying, the brother's spirit got trapped in Joshua's family ring. This is what allows Joshua to do magic, as his brother got turned into a sort of fire spirit that resides in the ring (thematically speaking). This is why most of Joshua's magic is fire themed.
He does not have direct contact with his brother's spirit, other than knowing it resides in the ring, thus why he is so protective of it. Joshua is constantly looking for ways to bring his brother back (magic, ethereal plane, begrudgingly working with Silence, etc).
The turning into the sparrow part when dying was Joshua being saved by his brother's spirit, which loosely represents a flame red sparrow when used for magic.