Listening test: WiiM Amp Pro vs WiiM Amp Ultra

[dominikz]

Hello everyone,

As promised in my thread with comparative measurements of the WiiM Amp Pro and WiiM Amp Ultra (link), this one will focus more on listening rather than the measurements.

TLDR; There are some test recording of both amps at the end for those who may be interested. I'd suggest to read the whole post for some context, but I won't blame you if this is too much to digest.

Initial listening impressions​

It took me a bit of time to figure out how to write this post without sending a wrong message.
So let me start by saying that most of the amplifiers I owned so far IMHO sounded good once loudspeaker placement, subwoofer integration and EQ was optimized.
One exception to this was a cheap amplifier that had pretty bad SNR which resulted in audible hiss at my listening position.

In line with that, I have to admit that subjectively I didn’t notice any issues in how the WiiM Amp Pro or the WiiM Amp Ultra reproduce audio. There’s no audible hum or hiss from either amp, both have sufficient power and don’t get anywhere close to clipping in my system, both have a subwoofer output with bass management, and both offer the same PEQ and RoomFit functionality. I didn’t have any issues getting my music to sound amazing with either of these amps, nor did I notice anything standing out with either (neither good nor bad).

Here I'm not trying to claim that "all amplifiers sound the same", because that is a generalization and oversimplification that simply cannot be true under every possible circumstance.
But I also don't believe that most amplifiers produce significantly different sound if used within their operating range (e.g. sufficiently below clipping/overload), and once basic variables (like output level) are controlled.

Having a little bit of experience with audio engineering, critical listening and psychoacoustics, I knew that to really be sure about my impressions I needed to prepare a more robust listening test - one where I can instantly switch between the two amps at will, and where relative playback level is always perfectly aligned.
Manually connecting and disconnecting amplifiers simply isn’t good enough to get any confidence in one's listening impressions.

Rigorous listening test attempts​

At the moment I don’t have a suitable hardware ABX amplifier switcher so I had to be a bit creative when preparing my tests. I came up with three different test concepts:

Listening test concept #1:​

• Connect the WiiM Amp Pro to the left loudspeaker
• Connect the WiiM Amp Ultra to the right one
• Level match both amps as close as possible using their individual volume controls while measuring the output level with a multimeter
• Group the amplifiers in the WiiM Home App, one as “L” the other as “R” channel
• Play music to this group and listen carefully if anything sounds off.

I’m happy to report that this worked well, and music sounded just fine - there were no obvious issues, and if I didn’t know any better I’d assume it was just one amplifier playing.

Listening test concept #2:​

• Put one loudspeaker on top of the other (but upside down), so they’d have similar acoustical interactions with the room and their tweeters would be closer
• Connect the WiiM Amp Pro to the bottom loudspeaker
• Connect the WiiM Amp Ultra to the top one
• Both amplifiers should play the same channel
• Level match both amps as close as possible using their individual volume controls while measuring the output level with a multimeter
• Group the amplifiers in the WiiM Home App so playback would be time-aligned
• Play music to this group, but always keep one of the two amplifiers muted.

Going in I was sceptical about this one because I suspected there would be too many sources of error - but decided to try it out anyway.

Unsurprisingly, this test didn’t work very well.

• First of all, the loudspeaker-room interaction (and therefore frequency response) differences between the two speakers were quite significant at the positions I tried.
• Secondly, I could easily hear whether the sound was coming from the top or bottom loudspeaker.
  • Because of the above two reasons the audible differences were also significant, and this was unrelated to the amp being used (i.e. it would be the same if both speakers were fed with the same amp) - this fact already made the test pointless.
• Thirdly, the WiiM Home App didn’t respond very reliably to all of the muting-unmuting. Trying to use the remotes didn’t help much with this either. This made the process quite frustrating for me.

So unfortunately this test concept didn’t pan out since the overwhelmingly audible differences didn’t have anything to do with the amps being used.

Listening test concept #3:​

• Connect the amplifier to the loudspeakers
• Disable the subwoofer output and EQ
• Level match both amps as close as possible using their individual volume controls while measuring the output level with a multimeter
• Stream a few songs to the amp and record the output signal at the amplifier output terminals with a high-performance ADC
• Do the same for the other amplifier
• Meticulously level-/polarity-match and time align the recordings.
• Use ABX listening test software to see if I can reliably and consistently hear any difference between the recordings made at the output of each amplifier.
• ABX compare each recording with the corresponding source audio files to see whether the amplifier is transparent or it introduces audible artifacts.

This test was naturally the most rigorous of the three.

Before going into my impressions let me provide more detail of the test setup process.

Listening test concept #3 - setup details:​

My main considerations when designing this test was that the test recordings should be made in a real-world usage scenario:

• The amplifier under test should be connected to a real set of loudspeakers
• In-room playback volume should be realistic and comfortable

The recordings weren’t made with an in-room microphone because that would add loudspeaker, room and microphone responses to the mix, potentially masking any real differences and making comparisons with original files impossible.

Instead the recording of the electrical signal at the amplifier output was made. This signal contains any distortions and deviations added by the amplifier to the original recording, including any caused by interactions with the connected loudspeakers. But these files are also suitable for reproduction on any standard audio system and can be compared to the original (source) files to test if the amplifier is transparent or not

Note that the complex impedance of the loudspeaker interacts with output impedance of the amplifier and causes a response deviation - something that may or may not be audible, but which doesn’t happen with resistive test loads. The extent of these response deviations depends mainly on how big or small is the difference between the amplifier output impedance and the loudspeaker impedance. More information and details on this can be read here and here.​

An extensive list of equipment used to create the recording can be found in the first post in the related thread with various measurements (link).

Here's an illustration of the basic recording layout:

Note: One difference with the equivalent WiiM Amp Pro layout was that I connected the Amp Pro subwoofer RCA sleeve to ground. An explanation can also be found in the related thread (link).

After the connections were made, I did a few basic measurements of the output to ensure everything works well:

• Play pink noise at -14 dBFS RMS and set output volume so that the level at the listening position 2m away was close to 70 dB SPL (Z-weighted).
  • Around 68dBZ SPL was measured at the listening position with volume set to "30" on the Amp Ultra and "34" on the Amp Pro.
• Play 1kHz tone at -14 dBFS RMS and check THD+N.
  • The results for the Amp Ultra were in line with baseline measurements. WiiM Amp Pro had some low-level spurious tones visible in the noise, but THD+N wasn't really affected.
  • Some increase of ultrasonic noise was visible with all amplifiers - in line with previous measurements and expectations.
  • In short - both amp's results looked OK.
• Do a sweep measurement and check frequency response, noise and distortion.
  • Results were in line with the baseline measurement for both amps.
  • The small frequency response deviations caused by the connected loudspeakers were visible, and both noise and distortion were negligible for both amplifiers.
• Check the file time alignment and potential clock drift.
  • Some very minor clock drift was visible, a bit more on longer recordings.

At this point I was ready for recording. I created a 192kHz / 24 bit recording project in PreSonus Studio One Professional 6.6 DAW, triggered stereo track recording from the E1DA Cosmos ADC Aux input and streamed several test signals and songs from my Samsung Galaxy S23+ phone local storage to the amplifier under test.

After the tracks reproduced through the first amplifier were recorded by the software, I disconnected it, connected the other amplifier and ran the same procedure again. For fun I did the same thing with my old Denon RCD-N9 amplifier.

In the end I had 4 parallel stereo tracks in the DAW SW:

1. One containing the original/source music files
2. One containing recording of the same music at the WiiM Amp Pro loudspeaker terminals
3. One containing recording of the same music at the WiiM Amp Ultra loudspeaker terminals
4. One containing recording of the same music at the Denon RCD-N9 loudspeaker terminals

The recordings were carefully time-aligned to significantly better than 1ms precision, and I used the -14dBFS 1kHz test signal as reference to roughly level match the relative levels of the tracks in the DAW software. The polarity of the WiiM Amp Pro recording was flipped to correct a known polarity-inversion fault (read about it here). I then used DeltaWave Audio Null Comparator software to fine-tune the level matching further.

Note that DeltaWave SW also confirmed the clock drift I saw visually in the DAW while time-aligning the files. At the moment I can't say if this drift is caused by the streaming protocol, the internal processing in WiiM DSP or by its DAC.​

At this point the recorded files could be exported.

Initially I exported the recordings as lossless FLAC files in full 192kHz / 24-bit resolution. Later I exported the same files in 48kHz / 24-bit as well.

Listening test concept #3 - my results:​

My ABX listening tests were performed on:

• My desktop nearfield loudspeaker system (RME Babyface into a pair of Neumann KH120A studio monitors and an Adam Sub7 subwoofer), and
• With headphones (RME Babyface into Objective2 headphone amplifier feeding Massdrop x Sennheiser HD 58X Jubilee Headphones)

After time, level and polarity matching I must admit that I could not reliably tell the two amps apart, and neither could I reliably pick out the original source file from the corresponding recording made at the output of the amplifier.

I did the test with both 192kHz / 24-bit and 48kHz / 24-bit resolution, with the same result.

From the perspective of audibility, both WiiM Amp Pro and WiiM Amp Ultra seemed to be transparent to the source to my ears.

Even the Denon RCD-N9 (which measures much worse) proved to be very difficult to pick apart - I might be able to do it with more practice, but I wouldn't bet on it. And in any case, if that much effort is needed the difference is obviously not that significant. Definitely not "night and day", not by any stretch.

Listening test concept #3 - links to a few recordings (with comments):​

Royalty-free test track attribution:

Music track: At Ease by Hazelwood
Source: https://freetouse.com/music
Vlog Music for Video (Free Download)

Note: I used other recordings for my tests as well, but I can only share royalty free recordings:

A note on resampling: Please configure your audio system to use 48kHz sample rate and 24 bit depth to match the test file resolution and avoid resampling.

Spoiler: Why you should try to avoid resampling:

In Windows 11 resampling from 48kHz to 44,1kHz will reduce the resolution by about 3bits, as you can see in these diagrams below. Note that the resampling artifacts are still below audibility, but they can easily be avoided altogether.
Note: Measurement signal level was -14dBFS RMS for both the 1kHz tone and multitone responses.

I decided against sharing the files in the format of a formal ABX test, and wanted to share them with some commentary and hopefully helpful explanations instead. Those who want to test their hearing acuity are welcome to use them in foobar2000 ABX comparator plugin (ideally in WASAPI Exclusive mode).

These files are suitable for listening on either headphones or loudspeakers - whichever you prefer.

1) For reference and comparison, here's the link to the original source file:

• At Ease by Hazelwood (source file, level-matched)

2) Now let's see how the same clip sounds after passing through the two WiiM Amps set to volume=30 but without level matching or polarity inversion:

• At Ease by Hazelwood (WiiM Amp Ultra)
• At Ease by Hazelwood (WiiM Amp Pro, original polarity and volume mismatch)

These two files represent what people would hear if they tried to set both amps to the same volume and switch between them - i.e. what probably happens in most informal listening tests people sometimes do.
Since both level and absolute polarity is different, the above two files should be relatively easy to pick-out in a blind listening test for most people.

3) Now let's level align the WiiM Amp Pro recording, but keep the original polarity:

• At Ease by Hazelwood (WiiM Amp Ultra)
• At Ease by Hazelwood (WiiM Amp Pro, original polarity but now level-matched)

These two files should be much harder to pick-out in a blind listening test for most people, and may be impossible to some.

4) Now let's also align the polarity:

• At Ease by Hazelwood (WiiM Amp Ultra)
• At Ease by Hazelwood (WiiM Amp Pro, polarity- and level-matched)

These two files should be the hardest to pick-out in a blind listening test - maybe even impossible.

Final thoughts​

For me this was quite an interesting exercise and I do hope it will be interesting to the community as well.

The measured performance of both WiiM Amp Pro and Amp Ultra is really good, and my listening tests indicate them to be transparent subjectively as well. Combined with the wealth of features and functionality on offer, relative ease of use, small form factor and great SW support I really can't fault these amps.

So I'd easily recommend either to those looking for an all-in-one device to use with passive loudspeakers - I'd choose the Amp Ultra if you have very low-impedance loudspeakers or want/need the display; otherwise the Amp Pro will likely work just as well (while costing significantly less).
The difference in grounding (two-pin vs three-pin power connection) between the two amps may be a factor to some people as well - depending on system layout and connections you might get a ground loop with one but not the other.

 

[adias]

And… you did not think the Amp Ultra was audibly significantly better than the Amp Pro? Because it definitely is. Any setup has a given audible sound signature - the sound atmosphere, instrument delineation, effortless crescendos, long decays. For anyone familiar with live acoustic sounds, there’s no contest between the 2 amps. Of course, YMMV.

Like I stated before, in my long audio experience, instantaneous switching tests are simply useless.

 

[dominikz]

And… you did not think the Amp Ultra was audibly significantly better than the Amp Pro? Because it definitely is. Any setup has a given audible sound signature - the sound atmosphere, instrument delineation, effortless crescendos, long decays. For anyone familiar with live acoustic sounds, there’s no contest between the 2 amps. Of course, YMMV.

Like I stated before, in my long audio experience, instantaneous switching tests are simply useless.

In my long audio experience, I respectfully disagree.

Also, note that having the ability to do instantaneous switching doesn't mean you can't listen to each track until the end and make your selection then - no one forces you to switch. The important part is to do the listening test with precise level-matching and under blinded conditions.

 

[adias]

If you are right all previous reviewers of the Amp Ultra are wrong. And that is not the case. The Ultra is significantly better than the other WiiM amps, except the Vibelink that is considered to be on par with the Ultra.

Clearly two different viewpoints - the objectivist vs the subjectivist. As long as each one is happy all is good.

 

[dominikz]

If you are right all previous reviewers of the Amp Ultra are wrong.

I'll be the first to suggest that anyone reading be reserved about any personal opinions I expressed so far.
But same goes for opinions shared by anyone else - including audio reviewers.

However I also shared data, references and detailed description of the whole process - which is something that could be relevant to other people. I also shared recordings that people can listen to and perhaps use to develop their own opinions.

This type of approach allows the proces to be replicated by other people, and findings to be validated or rejected.

If other reviewers share similar information too (even if they get the opposite findings) then we might have some common ground for discussion.

Without it we only have opinions, and we know that there are usually as many different opinions as there are people.
Opinions are of course fine, but not necessarily informative.

Clearly two different viewpoints - the objectivist vs the subjectivist.

I personally believe the whole subjective/objective thing in audio is a bit of a red herring, but I will admit to being a skeptic and a scientist at heart - for better or for worse.
In any case it should definitely be encouraged to share (and consider) different perspectives!

As long as each one is happy all is good.

Fully agreed!
People are free to have their opinions and to use the gear they enjoy (for whichever reason), there's absolutely nothing wrong with that!

 

[goodmore]

As an Amp Pro owner i have to say i was jelous of the screen on the ultra. But having the app on my fire tv has made me qutie happy. I look forward to WiiM expanding the capabilities of the App and maybe allow me to have total control of my WiiM network from my TV. Nice report on the comparison.

 

[dtc]

No one experiment like this is definitive for all users. The results depend on the nature of the speakers and the sound perception of the listener.

For example, some 3 way speakers have significant impedance variation throughout the frequency range and often have significant phase angle variance throughout their base range. Different amps handle these variances differently. Observations on a particular two way monitor does not necessarily apply to all other designs.

In addition, people have different hearing and different perception of what they hear. One individual's observations do not necessarily apply to all other people.

These are an interesting set of observations. But, in the end, they are one set of observations that should not be generalized to all cases.