At this time of the year it is more likely that you have attracted yellow jackets. The difference between the two is in how aggressive they are. Honey bees are docile and present only a minimal sting hazard. Yellow jackets are quite prone to stinging and will even chase the hummingbirds away.
Homemade Yellow Jacket Traps:
--Suspend a piece of fruit over a pan of soapy water, only an inch or two above. It helps if the pan or container is broad and shallow and for some reason, colored yellow. The wasps fly erratically, and if they drop into the water, the soap breaks the surface tension, so they can't "tread" on water to the edge of the container to escape. They drop in, sink, and drown.
---Plastic clear pop bottles with the tops removed and a 50:50 concentration of frozen apple juice concentrate, water, and a few drops of soap will have the same effect. Screw the lid on and dispose the bottle after a few days.
--Take a plastic gallon milk jug. About halfway up the jug, cut down slits into the jug and about 3/4 of an inch across and petal the edges inwards. Next, poke a hole, very small, into the bottle top. Feed a wire into it. Now, bend the wire, and jab a piece of salami, if they are still in the meat-seeking mode, or fruit onto the wire. Suspend the wire over the same mixture of juice, water, and soap, within an inch or so of the soapy juice solution. This can also be placed at ground level or suspended from grapevines, bushes, or fruit trees where the yellow jackets are most active.
--If you can, eliminate the fertile female in the spring, no later than May with the above traps. You can also use a piece of fresh meat that time of the year as a lure. If you do use meat, be sure that you change the bait every few days as Yellow Jackets, including the fertile female, will not be attracted to decaying meat.
Link to another good homemade trap. You will have to scroll up to the top of the thread.
Gardens Alive Yellow Jacket & Wasp Trap Excellent site for all sorts of things.
Leaving feeders up will NOT influence when healthy Ruby-throated Hummingbirds migrate south. Take most down by October1st, but try leaving one up until Thanksgiving (or even later if you can keep the mix from freezing); stray hummingbirds from the western U.S. may wander in and stay all winter (see Winter Hummingbird Research).