CFAO 55-10 -- The Creation and Awarding of Canadian Forces Flying and Specialist Skill Badges used to be the reference indicating criteria for permanent retention of such badges. I don't know if that order is still in effect, but it was likely superceded as per the forward to the current dress instructions
3. A-DH-265-000/AG-001 is effective upon receipt and supersedes all dress policy and rules previously issued as a manual, supplement, order, or instruction, except:
a. QR&O Chapter 17 – Dress and Appearance;
b. QR&O Chapter 18 – Honours;
c. CFAO 17-1, Safety and protective equipment - Motorcycles, Motor scooters, Mopeds, Bicycles and Snowmobiles; and
I think the previous version of the Dress Instructions included some of the detail that had been in CFAO 55-10 but it appears that the latest version has dropped much of that.
While CFAO 55-10 did provide instruction as to permanent retention of the various badges, it did so on the basis of each badge (or category of badges) outlining the criteria for award and as well as the criteria for forfeit of some badges (e.g. parachutist, EOD). I include (as illustration only, not as official reference) an extract from that CFAO - a copy I have from an old, out of date version on CD. Para 29 pertains to your question.
CLEARANCE DIVER BADGE
25. A CF clearance diver badge may be awarded to a member who has either:
a. successfully completed a clearance diving officers' qualifying
course and has been classified a Maritime Surface and Sub-
surface (MARS) (MOC 71D); or
b. successfully completed a clearance diver's qualifying course QL
5A and remustered to Clearance Diver (MOC 341).
26. Clearance diving officers and clearance divers are entitled to wear
the badge for the remainder of their service in the CF.
SHIP'S DIVER BADGE
27. A CF ship's diver badge may be awarded to a member who has either:
a. successfully completed a ship's diver officer's qualifying course
and been awarded OSQ BB; or
b. successfully completed a ship's diver qualifying course and been
awarded OSQ TT.
COMBAT DIVER BADGE
28. A combat diver's badge may be awarded to a member who has successfully
completed a combat diver's qualifying course and been awarded OSQ 45.B5, 041.23
29. Members awarded the CF ship's diver badge and the combat diver's
badge may wear the badge, subject to the following:
a. it will only be worn so long as the diver remains medically fit
and maintains the qualification by completing the required
periodic requalification courses;
b. short-term medical unfitness does not constitute grounds for the
removal of the badge; and
c. divers who maintain their qualification for a continuous period
of four years will be entitled to wear the badge for the
remainder of their service in the CF.