Oda explains that Hiyori made it to Onigashima during the Raid by hiding among Orochi's treasure tributes to the Beasts Pirates, intel she attained as an oiran. This is why she appeared at the Treasure Repository where the treasure tributes were left #ONEPIECE#SBS104
Oda clarifies that while we often see few members accompanying Shanks, the Red Hair Pirates actually have a lot of members, it's just that they don't always all move together. This is why Shanks's crew call him "Big Boss" (大頭) #ONEPIECE#SBS104