A circle is defined by three points

If you have three points, A, B, and C, then there is a unique circle that passes through those three points.

Define a function CIRCLE(A, B, C) so that it returns the center of the circle and the radius of the circle. Then CIRCLE(A, B, C) = (O, R). The order of input parameters doesn't matter. The result of CIRCLE(A, B, C) is the same as CIRCLE(B, C, A).

In Cartesian coordinates there are six input parameters and three output parameters, since each of the input points has an x and a y value, and the center of the circle in the output likewise has both x and y values. To restate: CIRCLE(Ax, Ay, Bx, By, Cx, Cy) = (Ox, Oy, R).

The three points define a triangle. The triangle ΔABC has side lengths a, b, c, a semiperimeter s, and an area K. These values are all useful when finding the center and radius of the circle that fits the three points. (By the way, the circle that fits the three points is the same as the circumcircle.)

The circumcenter's coordinates and radius are:

         (dA*(Cy-By) + dB*(Ay-Cy) + dC*(By-Ay))
 Ox  = -----------------------------------------   (1a) 
        2*(Ax*(Cy-By) + Bx*(Ay-Cy) + Cx*(By-Ay))

        -(dA*(Cx-Bx) + dB*(Ax-Cx) + dC*(Bx-Ax))   
 Oy  = -----------------------------------------   (1b) 
        2*(Ax*(Cy-By) + Bx*(Ay-Cy) + Cx*(By-Ay))

        a*b*c
  R  = -------                                     (2)
         4*K

where

 dA  = Ax^2 + Ay^2                                 (3a)
 dB  = Bx^2 + By^2                                 (3b)
 dC  = Cx^2 + Cy^2                                 (3c)
  a  = sqrt((Cx-Bx)^2 + (Cy-By)^2)                 (4a)
  b  = sqrt((Ax-Cx)^2 + (Ay-Cy)^2)                 (4b)
  c  = sqrt((Bx-Ax)^2 + (By-Ay)^2)                 (4c)
  s  = (a+b+c)/2                                   (5)
  K  = sqrt(s*(s-a)*(s-b)*(s-c))                   (6)

Example 1: Let A = (Ax, Ay) = (-3, -2), B = (Bx, By) = (6, 0), and C = (2, 7). Find the center and the radius of the circle that passes through A, B, and C).

Solution: CIRCLE(-3, -2, 6, 0, 2, 7) = (0.9, 1.7, 5.2). So the origin of the circle that fits the three points is O = (0, 9, 1.7) and the radius is R = 5.2.

 dA  = 13                                 (3a)
 dB  = 36
 dC  = 53
  a  =  8.1        Length of side BC of triangle ABC - bonus!
  b  = 10.3        Length of side CA of triangle ABC - bonus!
  c  =  9.2        Length of side AB of triangle ABC - bonus!
  s  = 13.8
  K  = 35.5        The area of triangle ABC - Super bonus!

I don't talk about a collinearity test. The CIRCLE algorithm really ought to trap for this condition. If two points are on the same line, then this algorithm will never work, the rocket you designed will crash, and you'll be boofed, you'll be fired from your high priced software engineering job, your girlfriend will leave you, and you'll be back to bagging groceries at the Winn-Dixie. Mea culpa? Nah, more like caveat emptor, baybee. If you're really a software guy, you should have developed some math skillz by now and know that you always need to check someone else's math before you use it in your own algorithm. Tough love, son.

Thanks to memplex for pointing out that if you assume one of the three points to be the origin (which is true for a lot of homework problems given in school), then a circle is specified by two points.

References

1. Equation of a Circle Given Three Points
2. Wikipedia, "Circumscribed Circle" This contains very useful mathematical formulae, especially matrix forms for finding the center of the circle, and exterior angles at the intersections of the circumcircle with the vertices of a triangle.
3. Weisstein, Eric W. "Circumcircle" From MathWorld--A Wolfram Web Resource.
4. Weisstein, Eric W. "Circumcenter" From MathWorld--A Wolfram Web Resource.