On this page of the game guide to Borderlands 2 you will find information about gaining upgrades, which can allow you to carry more ammo for your weapons.
After you reach the city of Sanctuary, you will be able visit Crazy Earl (square on the map) and buy upgrades to ammo capacity, weapons, grenades, and inventory and bank capacity. Each next upgrade of the same kind is more expensive. The capacity upgrades are however limited.
The currency for each upgrade is Eridium. It can be found pretty much everywhere: in lockers, ice, for killing enemies or completing missions. It's therefore worth to search the map thoroughly.
Finishing given missions (for example Plan B) unlocks further weapon slots. You can equip four of them at maximum.