What's New
Q & A
Tip Jar
C# Helper...
Follow VBHelper on Twitter
MSDN Visual Basic Community
TitleCompare floating point numbers safely
DescriptionThis example shows how to compare floating point numbers safely in Visual Basic 6.
KeywordsSingle, Double, compare, floating point, float
CategoriesAlgorithms, Software Engineering
Floating point variables hold approximate representations of numbers and sometimes they do not represent their values exactly. That means you cannot safely compare two floating point numbers to see if they are equal. While they may look equal to a quick glance, they may contain very small differences that make the program treat them as not equal.

The following code adds 10/17 to variable A 10 times so its value should be 10. It sets variable B to 10. If you look at their values, they look the same. If you subtract them, however, you find a tiny difference of roughly -0.0000009536.

The test A = B returns False because the values are not exactly the same.

The test Abs(A - B) < 0.00001 returns True because the values are very close.

The moral is, when comparing floating point values you should generally use a test similar to Abs(A - B) < 0.00001 (adjusting the size of the comparison value to suit your precision needs) rather than testing whether the two values are exactly equal.

Private Sub Form_Load()
Dim A As Single
Dim B As Single
Dim i As Integer

    For i = 1 To 17
        A = A + 10 / 17
    Next i
    B = 10

    lblA.Caption = Format$(A, "0.0000")
    lblB.Caption = Format$(B, "0.0000")
    lblAminusB.Caption = Format$(A - B, _
    lblAequalsB.Caption = CBool(A = B)
    lblAbsAminusB.Caption = CBool(Abs(A - B) < 0.00001)
End Sub
Copyright © 1997-2006 Rocky Mountain Computer Consulting, Inc.   All rights reserved.