Leaflet fitBounds inconsistency

Hello,
I had a past question around creating consistent sized maps around output areas covered by GP practices. This was solved using fitBounds when I wanted there to be consistency in the size of the black boundary layer. However, I am finding that when I use fitBounds on a second polygon I can get too much space around it. I want my map to be as large as possible. See examples:

The first map has fitBounds applied to the surrounding polygon, the black line. The second has fitBounds applied to the coloured thematic showing the catchment area where patients live. I want the second map to be as large as possible. Is there any way I can do that or is it down to this:

Note though that the zoom level is discrete, not continuous. So there will be some zoom levels that are not achievable.

The code I am using to create the second map is:
m <- leaflet(j) %>% addTiles() %>% addPolygons(fillColor = ~pal(Patients),
fillOpacity = 0.5, color = "#000000", weight = 1, opacity = 1) %>%
addPolygons(data = ccg_br, color = "#000000", weight = 5, fillOpacity = 0) %>%
addAwesomeMarkers(lng = coord$long, lat = coord$lat) %>%
addLegend(pal = pal, values = ~Patients) %>%
fitBounds(lng1 = unname(bbox_j[1]),
lat1 = unname(bbox_j[2]),
lng2 = unname(bbox_j[3]),
lat2 = unname(bbox_j[4]))

j is my output area spatial polygons dataframe while ccg_br is the black line boundary. I am saving the maps as .png for use in a report.
Many thanks