Ambient Sounds

[Fixer-007]

I know there are ambience mods already, but it is little strange to play default in totally quite with just foot steps and axe sounds.

I think default needs at least 3 additional sounds (with slight volume):

• winds in farm env
• winds in cold env
• liquids
• not sure about droplets in caves

I was browsing freesounds.org and found pretty good records of wind like those, that can be a good source:
http://www.freesound.org/people/felix.blume/sounds/156414/
http://www.freesound.org/people/Dynamicell/sounds/17553/
http://www.freesound.org/people/Leandros.Ntounis/sounds/163607/
http://www.freesound.org/people/felix.blume/sounds/165526/

Licenses are CC Attribution License and CC 0 License.
What do you think?

[rubenwardy]

Related: minetest/minetest#2272

[kilbith]

Not conceivable without client-side modding (i.e. not before a long time).

Globalstep is bad for that since the player position and the sound's triggering are often not perfectly synchronized, and it can be a factor of lag.

[paramat]

Only short occasional sounds can be used due to huge file sizes, so not wind, it's only a steady 2m/s anyway. There have been attempts at flowing water sounds but there were problems, maybe an occasional sound from flowing water, but has to not get heavy with large numbers of flowing nodes.

[0-afflatus]

I can't think of any way wind could be done. Ambient sounds would need some kind of locking whereby each sample could only be played by a limited number of nodes (the nearest to the player) at a time. The other option is to allow players to place sounds independently of other nodes. That may be do-able without an engine change and something I might look at once grailtest is stable.

[paramat]

Sound of wind itself makes no sense as it's only 2m/s, the sound of rustling leaves in a gentle wind makes more sense. This would need checks on light level being 15 to ensure the leaves are 'outside'.
Let's not add sounds suggesting something not actually happening in a Minetest world.
Additionally flowing water sounds and a new method for fire sounds are needed.

[sofar]

Client side modding would certainly be able to address this, but even an engine change could solve this problem:

• Just as the client code plays footsteps, it could scan the immediate area around the player and add static sound loops without lag or packet overhead.
• This would allow fire sounds to be played, or water sounds, or leaves rustling in the wind
• The sounds could be added to the nodedef.sounds table which is already serialized and sent to the client, so no protocol change is needed (afaics)

I think ultimately this shouldn't be part of minetest_game, so this should be closed.

[paramat]

Rubenwardy is intending to look into new engine support for ambience sounds.
Until then the method of tenplus1's ambience mod has potential for replacing the problematic fire sounds (maybe for flowing water too) see #1112
Perhaps this issue is valid to keep open then?

[Fixer-007]

Sound of wind itself makes no sense as it's only 2m/s, the sound of rustling leaves in a gentle wind makes more sense.

That's what I was talking about :) I disagree about wind near trees only, you can still hear wind in deserted areas and in mountains (a lot), ocean... everywhere, it is just different kind of wind, don't make other places windless.

[0-afflatus]

I'm looking at tenplus1's ambience mod atm. That may be the WTG.

[paramat]

you can still hear wind in deserted areas and in mountains (a lot), ocean... everywhere

Only if it a strong wind. 2m/s is not fast enough.

[Fixer-007]

2m/s

How do you know it is 2m/s? On mountains/ocean/beach it should be way higher

[paramat]

I measured cloud speed by timing a cloud over a distance. It's a very gentle wind.

[rubenwardy]

That's hilarious

[Fixer-007]

I measured cloud speed by timing a cloud over a distance. It's a very gentle wind.

(after some thinking) We need to increase cloud speed for sure...

[paramat]

Nah, they're slow because they're close, it's to preserve a feeling of serenely drifting clouds, at a realistic speed they would visually move too fast.
Clouds are roughly 10 times lower than realistic, so the visual speed is actually that of clouds at 1280m moving at 20m/s or 45mph.
Perhaps in future we can make cloud speed a setting.

Ambience sound is heavy to implement so the less we have of it in the basic game the better, so just fire, flowing water, gently rustling leaves at the most.

[paramat]

Leaves in wind will need a light level check (level 15) to make sure they are 'outside'.

[sofar]

According to http://www.spc.noaa.gov/faq/tornado/beaufort.html, wind speeds roughly are:

beaufort 4 : ~8m/s
beaufort 5: ~11m/s
beaufort 6: ~14m/s
beaufort 7: ~33m/s
beaufort 8: ~20m/s

---

Anything under 4 is not enough to fly a kite and probably should not have wind sounds. 5-6 you get some rustling leaves and branches, especially in high trees. 7-8 are loud.

The problem with minetest clouds is that they only have one speed. If the wind sounds would always play outside then that would be unbalanced.

[C1ffisme]

@sofar Maybe a weather system? (Just wind used for clouds, detectable by mods for now.)

[paramat]

SInce waving enabled makes stuff sway gently i think we can assume a ground wind speed above 2m/s, and add gentle leaf sounds consistent with the swaying.

[DonBatman]

Imho if you add wind it would need to be more than a 5 second sample. Imagine playing for an hour with the same sound playing 1800 times. It would need to be random and play occasionally.

[paramat]

For now i suggest we consider using an edited version of tenplus1's ambience mod for flowing water and fire.

[sorcerykid]

The past few weeks I've been developing a mod for environmental sounds in Minetest. I'm hoping to have a proof of concept working by next week on my JT2 server. I can't go into any specifics about the implementation yet, since the mod is still in the alpha stage. But my intention is to have a very lightweight and extensible framework for ambient, positional, and interactive sounds, particularly for use in multiplayer. I've modeled some my ideas off of the Dark Engine, which was used in Thief and Thief 2. Both featured some of the most compelling and immersive soundscapes of any 3D computer game up until their time (and for good reason, since sound and light were integral to the gameplay mechanics).

Once I actually have a beta version ready for release, I will submit it in the Minetest modding forum.

[MarkuBu]

My ambiance mod uses ABMs which are very lightweight, but I use a lot of long samples. With just a few short samples we would have repetitive effects.

What we need is a client side cache that media files only need to download once

[rubenwardy]

Client side modding already supports get_node() and play_sound(). It should be possible to rig up zero latency ambient mod there (Although media loading hasn't been implemented afaik, you'll need to provide the sounds via a server mod)

[sorcerykid]

I'm not entirely convinced that client-side modding is the end-all, be-all solution. There still needs to be communication with the server to ensure that sounds are consistent for all players. That is how visuals work, after all. When a portion of the map changes, all clients within range are notified about the updates dynamically. So I would argue that audio should function mostly the same way. To have one client play a sound, but other clients play a different sound without basic synchronization, is not particularly realistic.

If the implementation is done well enough, there should be very little overhead on the server side. I believe the ambience mod by sfan, continuously analyzes the map at certain intervals, then plays or stops sounds at random positions accordingly which can be quite taxing. My mod, however, relies on a global sound matrix state-table. So map analysis is only performed when blocks are loaded or the cache entry expires. Sounds are played in active areas of the map and stopped in inactive areas of the map automatically via server active objects (with some additional heuristics for load-management).

[rubenwardy]

that sounds are consistent for all players

Not necessary for things like waves/wind/rustling

Event sounds like chests/birds/etc should be done server side

[MarkuBu]

To be honest it doesn't makes sense to play all ambient sound from all players to all players. If there are several players in the same area it could be very noisy.

I have the same issue with my mod, that's why it currently wouldn't work on server. My mod plays sounds from a position, which makes it special because you can hear directional sounds.

[sorcerykid]

I'm not sure. My mod employs positional sounds for ambience. But I haven't encountered any issues with being noisy, even in multiplayer.

[Ferk]

When a portion of the map changes, all clients within range are notified about the updates dynamically. So I would argue that audio should function mostly the same way.

Wouldn't client-side ambient sounds be based on the map information? since the map changes are notified to all clients, ambient sounds would already be using the same info for all players in the same area.
Imho a client-side ambient sound mod makes perfect sense.

[sorcerykid]

Indeed, but I think localized and non-localized sounds should not necessarily be tied to single nodes or active objects whether for backend storage or client-server updates. It should be possible to define ambient and/or random sounds based entirely on environmental factors. Yet this is not at currently possible, without an engine change. My favourite example is the random frog croaking.

[Ferk]

based entirely on environmental factors

The thing is that it's hard to picture an environmental factor that is not associated to node / object information. Or even server time / seed / location.

How does the server know that an environmental factor is happening and why can't the client be able to know this as well if it has all this information from the server already?

If you code weather effects entirely in the client based on factors that are shared by all clients then it follows that the weather effects would be the same for all clients.

[sorcerykid]

I can think of numerous instances where a sound should not be attached to a specific node or object. Random birds chirping or wind blowing might have nothing do a single node or object, but rather with the overall environment. In other words, I might only want a frog to croak in a map block with a 1/60 chance in which there is at least 500 default:water_source nodes with air nodes above and at least 200 default:dirt_with_grass nodes with air nodes above, an altitude of -32 to 64, and between the game time of 0.2 and 0.8 with a gain of 0.5 and a maximum range of 50 using linear attenuation.

How would such a sound definition be stored within the database on the server side and transmitted to the client during a map update and then automatically played back to all clients simultaneously?

[MarkuBu]

I use ABMs

[sorcerykid]

ABMs are less than ideal because there is no fine-grained control over the number of sounds being played in a specific area of the map at any given time, which is critical in a multiplayer.

[Ferk]

Random birds chirping or wind blowing might have nothing do a single node or object, but rather with the overall environment

I wasn't talking about "a single" node or object.
If the "overall environment" is determined by node information, objects, time, location and/or seed, then it can be done client side. You can do the client side checks you want and if appropriate play a sound to the player without it being necessarily attached to a particular node/object and without using up server and network resources.

So long as all the clients have the seed, time, object and mapblock information, they'll be able to hear the same sound.

[paramat]

I'm working on a simple ambience mod for water sounds, fire sounds will be combined into it.
I realise this "isn't enough" but it will be temporary until we get better engine code for ambient sounds, then the mod will be the place for the improved code.

[sorcerykid]

If the "overall environment" is determined by node information, objects, time, location and/or seed, then it can be done client side.

You raised excellent points Ferk. My soundscapes mod uses a heuristic approach for environmental analysis, and then caches the results within each mapblock. While the proof of concept is server-side, it is nonetheless very efficient and lightweight. Porting to CSM, would require only a few alterations, so it might be worth investigating if there is the potential for additional performance improvements.

[ghost]

Now we have Client Side Mods and I think now we can try to do it?

[paramat]

We don't have server-sent CSM yet though.

See #2064 as an example of how to do water sounds.

[paramat]

#2064 merged. A simple start for the short term, to be improved later once new engine features are added.

[paramat]

#2683 merged, so all liquids are done, which were the most important sounds to add.
Closing due to #2710