First and most importantly you have already told this fellow about your status. So even if you haven't had contact since, he is aware that if he ever develops symptoms he should seek treatment. He is also an adult capable of deciding whether he will risk that you might be shedding at some point in time when you are fooling around/having sex.
Second. HSV1 is more difficult to transmit when it's located in the vaginal area. It doesn't like to shed from there so the odds are that it is unlikely you were shedding at the time of your bit of fun.
Third. If you did not feel actual OB symptoms down yonder (especially with HSV1) its not likely you were shedding. Having an OB after intercourse (if that's what you've got going on) is not unusual as the irritation to the area can sometimes trigger an OB.
Fourth. The people who wind up with herpes whitlow usually get it because they are shoving their hands in the MOUTHS of people with cold sores, skinning their fingers/knuckles on those peoples teeth, and getting exposed that way. There are some exceptions, but it is an uncommon area for people who are not wrestlers, dental hygienists, or dentists to contract it.
Fifth: As you know there is a high likelihood your friend/lover already has HSV1 orally from childhood. Of course you can't assume that, but if he does have it (even if he never remembers having a cold sore he could still have it) it is very unlikely he would catch it from you in any location.
Try to relax. It's pretty unlikely he contracted anything from you under the circumstances. Keep breathing and repeat as necessary
If you are good terms with him and you can always tell him you've been feeling run down and you wanted to see how he is doing. But you are not obligated to call him.
Hoe eet jy 'n olifant? Bietjie vir bietjie.
(Translation from Afrikans: How do you eat an elephant? Bit by bit.)