My Ultra tests

[steadyshot]

More issues, too.

The dongle isn’t a DAC. It’s a DAC/headphone amp, with only a headphone out, so after the DAC the signal has to go through a very (very, very) cheap headphone amp.

No one even remotely interested in conducting a properly controlled test would ever do that.

These are the audibility thresholds suggested by a member on the asr forum on the famous thread. Can we map the measurements to these suggested thresholds? If anyone could find them...

 

[Steve Woodhouse]

How good must the Apple dongle measure to be transparent , then ? Give me a number please.

"High-End" DAC Blind Listening Results - PART I: Devices Unveiled!

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

View attachment 9126
View attachment 9127

As already noted, the dongle is als a headphone amp. It performs very well with low impedance phones, but poorly with high impedance.

For the test to mean anything (and measurements) the listeners would all have to specify the volume they set their iPhone/Pad to, and then measure that. Because you’ll get different results (both in listening and measurements) with different volume levels.

 

[Jls]

It shows there was differences in sound. Wether real or only feeled can't be said.

Did you run a p-test on that data?

 

[Steve Woodhouse]

View attachment 9128

These are the audibility thresholds suggested by a member on the asr forum on the famous thread. Can we map the measurements to these suggested thresholds? If anyone could find them...

Yes, THD+N is about 98dB, so just above lenient, way short of strict.

Dynamic range inbetween the two (but lots better) at 113.

Jitter is worse than 120 in some parts - not by a lot, but over a broad range (7kHz to 19kHz).

These are still excellent specs - astonishing for $10, and close to transparency.

But as I say, unless we know what every participant had their volume set, it’s meaningless.

Again, as we have nothing but remote listener’s word for the results, meaningless.

And once again, the results were far from conclusive. Effectively, let’s just call the results ‘right and ‘wrong’ for a moment. Out of 105 tests adding the two high ends together, the participants only picked the ‘right’ answer (most expensive set ups as best) 60 times, with 45 choosing the single or no difference.

That’s 57% vs 43%. No one could possibly thank such a narrow result could prove anything.

 

[Jls]

In

In that test 26 preferred the $10 dongle, 29 preferred the $3000 streamer, 31 preferred the $20000 streamer and 19 had no preference. What does that show?

The two-tailed P value equals 0.9242
By conventional criteria, this difference is considered to be not statistically significant.

Answer:
z score
z = 0.095202121529773
Sample Mean = 26.25
Count
n = 4

 

[onlyoneme]

"Check this out - for the listeners who thought they could hear a difference, let's separate them by whether they used headphones, speakers, or both"
"Excellent! That pattern among the headphone listeners is highly significant with χ2 = 18.9, p-value = .00084; clearly much better than the typical p < .05 significance threshold!"

 

[onlyoneme]

View attachment 9128

These are the audibility thresholds suggested by a member on the asr forum on the famous thread. Can we map the measurements to these suggested thresholds? If anyone could find them...

None of the WiiM devices can pass all these strict thresholds. I'm not sure if such a device even exists considering presence of IMD here, although undefined.

 

[Steve Woodhouse]

"Check this out - for the listeners who thought they could hear a difference, let's separate them by whether they used headphones, speakers, or both"
"Excellent! That pattern among the headphone listeners is highly significant with χ2 = 18.9, p-value = .00084; clearly much better than the typical p < .05 significance threshold!"

Yes I saw that. To be clear, a sample size of 105 is too small for a definitive conclusion, even if the results were clear cut (which they clearly weren’t).

Once you only check headphone listeners, all the more so.

 

[Steve Woodhouse]

None of the WiiM devices can pass all these strict thresholds. I'm not sure if such a device even exists considering presence of IMD here, although undefined.

For clarity. At the moment we’re only talking about the Pro Plus (Mini and Pro aren’t transparent, Amp has no line level analogue out from digital).

So, Pro Plus. THD+N c.115dB, so extremely close to ‘strict’. Dynamic range c.117.5dB, again extremely close to strict. Jitter always below 130dB, so beats strict.

So if we take strict as a target, the dongle doesn't beat any, and falls quite short with THD+N, whilst the Pro Plus clears one, and is within a whisker on the other two.

Of course, there are genuine and legitimate debates about the limits, no question.

Just one more thing. Whilst I have no measurements for the more expensive Linn, the Majik scores THD+N 95.5dB. That’s the only measurement I have from the digital out. SNR is abysmal at 69dB.

So it looks like they were comparing DACs which aren’t transparent.

 

[Brantome]

That’s


Yes, I saw that.

Once you


Yes I saw that. To be clear, a sample size of 105 is too small for a definitive conclusion, even if the results were clear cut (which they clearly weren’t).

Once you only check headphone listeners, all the more so.

Why the three quotes?

 

[steadyshot]

For clarity. At the moment we’re only talking about the Pro Plus (Mini and Pro aren’t transparent, Amp has no digital out).

So, Pro Plus. THD+N c.115dB, so extremely close to ‘strict’. Dynamic range c.117.5dB, again extremely close to strict. Jitter always below 130dB, so beats strict.

So if we take strict as a target, the dongle doesn't beat any, and falls quite short with THD+N, whilst the Pro Plus clears one, and is within a whisker on the other two.

Of course, there are genuine and legitimate debates about the limits, no question.

Just one more thing. Whilst I have no measurements for the more expensive Linn, the Majik scores THD+N 95.5dB. That’s the only measurement I have from the digital out. SNR is abysmal at 69dB.

So it looks like they were comparing DACs which aren’t transparent.

Do you know how many dacs pass all the strict thresholds and how much does the cheapest of them passing all strict cost?

 

[Jls]

Just make sure you aren’t one of those who think the difference is absolutely vital, and prefer the $10 DAC.

 

[onlyoneme]

For clarity. At the moment we’re only talking about the Pro Plus (Mini and Pro aren’t transparent, Amp has no digital out).

So, Pro Plus. THD+N c.115dB, so extremely close to ‘strict’. Dynamic range c.117.5dB, again extremely close to strict. Jitter always below 130dB, so beats strict.

So if we take strict as a target, the dongle doesn't beat any, and falls quite short with THD+N, whilst the Pro Plus clears one, and is within a whisker on the other two.

Of course, there are genuine and legitimate debates about the limits, no question.

Just one more thing. Whilst I have no measurements for the more expensive Linn, the Majik scores THD+N 95.5dB. That’s the only measurement I have from the digital out. SNR is abysmal at 69dB.

So it looks like they were comparing DACs which aren’t transparent.

Presence of the digital outputs is irrelevant here.

And for clarity, I specified already criteria to be fulfilled based on the discussion already if you want to continue it with me here. There is no "we" so far, and that's your trolling instead.

 

[Steve Woodhouse]

Why the three quotes?

Apologies - corrected. It was completely unintentional

 

[Steve Woodhouse]

Presence of the digital outputs is irrelevant here.

Look at any amplifier review, and it’ll have measurements for noise and distortion worse than almost every DAC. You can’t measure a DAC fairly by only measuring it after it’s been through an amplification section. If the amp measures worse than the DAC, you’re measuring the amp, not the DAC (unless the DAC is worse than the amp).

I’m sorry, I thought that would be blindingly obvious.

EDIT - for clarity I’ve edited that to read ‘line level analogue out from digital’.

 

[Steve Woodhouse]

Do you know how many dacs pass all the strict thresholds and how much does the cheapest of them passing all strict cost?

Good question.

Starting at the lower end ,Topping D10 balanced, THD+N 118.5dB (extremely close) SNR 121.5dB (pass), jitter 139dB (pass).

The DX7Pro Plus clears everything. By the way, in relation to onlyoneme’s comments on the WiiM Amp (above), the SNR is 130dB on the balanced out, but only 93dB on the headphone out.

That’s why it’s simply not possible to measure the DAC alone on that model.

 

[onlyoneme]

@Steve Woodhouse
There is a very special kind of the ANALOG outputs built in the DACs, due to the digital to ANALOG conversion function, being used for connections to the amps for example. And also for measurements of the performance of digital to ANALOG conversion.
Well...

 

[Steve Woodhouse]

@Steve Woodhouse
There is a very special kind of the ANALOG outputs built in the DACs, due to the digital to ANALOG conversion function, being used for connections to the amps for example. And also for measurements of the performance of digital to ANALOG conversion.
Well...

You probably need to read my edit.

You cannot measure a DAC accurately from speaker level outputs.

 

[Smartplug]

I’m sure I will get hammered on this especially Steve Woodhouse who like busting chops to other users. I have 7 dacs and all modern devices. 3 uses Ess chip different model and using same opa1612. 3 also uses akm different model and using opa1612. Both chipsets measure extremely good. Now, I could not hear the difference version of ess or akm but I could easily hear if it’s akm or ess chipset. How did I do this? I had one guy cover my eyes and sent me to other room while switching dacs through a switch then guide me to the chair blindfolded and ask me which chipset while playing music. We did this 50 times and I got zero error. I am able to pinpoint which chipset being played between ess and akm. Now, if measure identical should sound same so how did I able to tell the difference? Golden ear!

 

[alpha_zulu]

I’m sure I will get hammered on this especially Steve Woodhouse who like busting chops to other users. I have 7 dacs and all modern devices. 3 uses Ess chip different model and using same opa1612. 3 also uses akm different model and using opa1612. Both chipsets measure extremely good. Now, I could not hear the difference version of ess or akm but I could easily hear if it’s akm or ess chipset. How did I do this? I had one guy cover my eyes and sent me to other room while switching dacs through a switch then guide me to the chair blindfolded and ask me which chipset while playing music. We did this 50 times and I got zero error. I am able to pinpoint which chipset being played between ess and akm. Now, if measure identical should sound same so how did I able to tell the difference? Golden ear!

Very interessting. Did you use always the same track? Or can you do this with any random song? How would you subscribe the main differences between ESS and AKM?