There are a few different answers – depending on what type of fish we are asking about.
The most sophisticated types of fishes, bony fishes, have a swim bladder. These fish can inflate their swim bladder with gas from a special gas gland. The gas is basically oxygen from the fish’s blood. Bony fishes that spend most of their lives on the bottom of the ocean floor (e.g., flounder) don’t have a strong swim bladder – therefore, don’t float.
Sharks, skates, and rays are all types of fish. But, they are more primitive. They stay afloat with a fancy liver filled with oil, using their long pectoral fins for balance in mid-waters, and maintaining a light framework. The ‘light framework’ is made up of cartilage (the same stuff found in our nose and ears).