As a consumer
Your options are often quite limited as a consumer, but you can minimise your risks in a few ways through carefully selecting the products you use and how you connect them.
Check what happens when your device loses Internet connectivity
Usually, you can just do a quick Google search to see what happens when a certain device disconnects from the Internet. Some devices will simply fail completely if their connection to a remote cloud server is lost, like the Amazon Echo:
Your Echo requires an active Wi-Fi connection to speak, process your commands, and stream media.
Sometimes, there's a good reason (for example, the Echo has to stream commands to the cloud to process your instructions, as stated in 'Is the Amazon Echo 'always listening' and sending data to the cloud?'), but for others, it may just be an oversight or design flaw in your product.
If you physically have the device, you could try unplugging your router to see what happens—this might not be a great test, because it's more likely a remote server will break but local connections will still work, but it's something to try.
With enough time to waste use productively to improve your setup, you could potentially sniff packets from your devices, then apply a router-level block to certain domains—this way, you'd know what happened if mydeviceserver.com went down completely. Of course, this would take a long time so it might not be practical to test all of your devices in a large home with lots of 'smart' devices.
Use local connectivity
If you're just turning your lights on from your smart switch, you might not need to route all the traffic through the Internet, into a cloud server thousands of miles away, and back to your lightbulb—you might just be able to route the command through local devices instead. A lot of the time, these devices will use a protocol like ZigBee or Z-Wave, so you might need a hub to co-ordinate the traffic (see 'Why do I need hubs for some devices when automating my home?').
As a developer
For developers of IoT devices, careful design of a device can avoid problems like the recent S3 outage from affecting consumers. Of course, IoT designers aren't always known for careful design, but if you're reading this, you're probably not in that group.
Design services to be redundant
For Amazon S3's recent outage in particular, there may not have been much you could do. There are some reports that cross-region replication could have potentially prevented services from going down, as explained in this question on DevOps Stack Exchange, but it's debated whether that's really true or just poor advice.
If feasible, having some sort of redundancy or backup would be ideal—although the costs are greater, the additional reliability is greatly needed—otherwise, people's lights stop responding, power switches refuse to work, etc.
Add better support for scenarios without Internet connections
I listed 'Use local connectivity' under the ways that a consumer could avoid this issue, but it's a losing battle. The devices often don't support connecting in any other way than through their approved web service, and manufacturers are reluctant to spend developer time on this. If the support was greater, there would be less reliance on cloud services, which benefits the manufacturer too, because they don't need to pay for so much server capacity.
With all these options, why were so many devices affected?
Because no-one wants to spend the time—designing any sort of reliable system takes a lot of time and effort, and it's often far more complex than the comparable 'dumb' solution (e.g. simple electrical switches).
Why isn't software as reliable as a car? Because the software has so much more complexity, yet isn't tested nearly as rigorously as a car would be. The same issue seems to apply with IoT—controlling devices through a network is far more complex, so things can go wrong far more easily, as we've seen with the recent S3 incident.
