How precise are measuring instruments?

Topics include, Machine Tools & Tooling, Precision Measuring, Materials and their Properties, Electrical discussions related to machine tools, setups, fixtures and jigs and other general discussion related to amateur machining.

Moderators: GlennW, Harold_V

Mr Ron
Posts: 2126
Joined: Tue Dec 22, 2009 12:36 pm
Location: Vancleave, Mississippi

How precise are measuring instruments?

Post by Mr Ron »

The question has come up as to how square is a table saw blade to the table. If you want the blade to be exactly 90°square to the table, you would use a machinist's square and check for any light (gap) between the square and the blade. The question is; how precise is a machinist's square? Is there a ± value on a square? Can one say a square is 90° ± .xxx°. Precision tool manufacturers don't seem to publish accuracy. The same is also asked about length dimensions. Is a micrometer that measures to a thousand actually .000 or is it ± tenths?
Mr.Ron from South Mississippi
User avatar
rmac
Posts: 787
Joined: Tue Jun 26, 2012 12:48 am
Location: Phoenix, Arizona

Re: How precise are measuring instruments?

Post by rmac »

At least some makers of precision measuring equipment do publish accuracy specs for their products. For examples, poke around on Starrett's website to see how they describe their squares and micrometers.

-- Russell Mac
User avatar
GlennW
Posts: 7284
Joined: Sun Apr 15, 2007 9:23 am
Location: Florida

Re: How precise are measuring instruments?

Post by GlennW »

Checking to see how square a square is.
viewtopic.php?f=4&t=95242&p=268400&hili ... re#p268400
Glenn

Operating machines is perfectly safe......until you forget how dangerous it really is!
User avatar
Bill Shields
Posts: 10460
Joined: Fri Dec 21, 2007 4:57 am
Location: 39.367, -75.765
Contact:

Re: How precise are measuring instruments?

Post by Bill Shields »

I use a builders square and check for gap at the front and back side of the blade. Then I turn the square over and check that the result is the same.

Sure .the builders square is not perfect but being the size that it is makes it convenient to work with.. and turning it over and checking from the other side ensures that the average is square.
Too many things going on to bother listing them.
atunguyd
Posts: 199
Joined: Tue Oct 20, 2009 9:39 pm
Location: Durban South Africa

Re: How precise are measuring instruments?

Post by atunguyd »

At the risk of being pedantic, in your initial question you keep flip flopping between accuracy and precision. These are two different concepts.

A instrument can be very precise but completly inaccurate.

To put it simply precision is the number of decimal places. So a ruler that only measures to mm is not as precise as a micrometer tubs measures to micro meters.

But if my ruler says that the 10cm block is 10mm then it is accurate. Whereas if my micrometer says that the same block is 10.18mm it is still more precise than the ruler but not accurate.

Sent from my SM-N975F using Tapatalk

User avatar
Harold_V
Posts: 20231
Joined: Fri Dec 20, 2002 11:02 pm
Location: Onalaska, WA USA

Re: How precise are measuring instruments?

Post by Harold_V »

atunguyd wrote: Sat Dec 25, 2021 11:29 pm A instrument can be very precise but completly inaccurate.
Correct!
As an example, a micrometer that reads tenths may consistently measure an item at a given size, but the indicated size isn't the true size of the item being measured. It would be akin to a micrometer that is set off by a couple thou---it will repeat the reading (consistently), but the reading isn't correct.

On the subject of making measurements--the surface finish of the item being measured plays a huge role in the veracity of the reading. A ground surface is more likely to be measured properly than one that has been milled, as an example.

As for the ability of measuring instruments, all of my outside micrometers are made by Starrett---each is tenths reading, equipped with carbide faces. When I was in precision grinding, I used my 1" micrometer daily, as the bulk of the parts ground were under 1" diameter. We routinely used a 100 grit wheel, so surface finishes were superb. I had developed a feel that allowed for measurements within .000050" consistently---with a Sheffield Shadowgraph at my disposal to ensure my readings were correct (I've related this before on this board). That should give you an idea of the capability of a decent micrometer.

H
Wise people talk because they have something to say. Fools talk because they have to say something.
User avatar
liveaboard
Posts: 1971
Joined: Sun Dec 08, 2013 1:40 pm
Location: southern Portugal
Contact:

Re: How precise are measuring instruments?

Post by liveaboard »

You need to start with the flatness of the saw table and the blade. Also whether the arbor is true.
If the saw fence isn't parallel to the blade, cuts won't be right either.
Cross cut miter too.
Then the wood springs as you cut.

When I was looking for a square builder's square (in the Netherlands, 1990), I went to a wholesaler who does retail, with a machinist square. I went through his stock shelf until I found a few. 90% had errors I could see by eye.
Those were (supposedly) euro made squares. They appeared to be ground.

A ground straight edge is vital too, and they're not cheap either.
pete
Posts: 2518
Joined: Tue Feb 10, 2009 6:04 am

Re: How precise are measuring instruments?

Post by pete »

Your questions aren't that simple to answer Ron. But fwiw, it depends on the level of quality a buyer is prepared to pay for if you'll get any metrology equipment that comes with any sort of "believable" guarantee of accuracy or not. Since nothing made is ever perfect there's a recognized allowable fudge factor. Secondly is if the manufacturer is trustworthy enough to produce and back up with proper quality control checks all of there products to meet those recognized standards. There's some off shore produced metrolgy equipment that I've seen that sure as hell doesn't meet the industry world wide standard of + - one count for repeatable accuracy even though the paperwork says it should. And unless I check it myself, I don't blindly believe any paper work from any company. Mistakes can and do get through the best quality control checks in every single company. Although so far all my Mit. and Starrett equipment has passed any checking I've done, they'll generally hit at around half that allowable tolerance. But it's impossible to know that unless you do check. "Good" .001" reading micrometers have to be within + - .001" to pass as within that allowable tolerance limit. A .0001" reading mike has to be within + - .0001" to pass QC. Today and for the little extra they cost, I'd buy a 10ths reading mikes just for the better QC and easier to read accuracy even if I never planned on measuring anything down to actual 10ths. In reality I doubt the internals on .001" or .0001" mikes from the same manufacturer are any different though. So the extra cost is just for the vernier markings. For the better metrology equipment that + - one count is the allowable limit that still passes.

The paper work that comes with any decent measuring equipment always has 3 terms that too many don't fully understand. #1 Accuracy, that's a combination of the quality of all the tools components and the accuracy it was built and can allow it repeat to. #2 Repeatability, that's directly related to the tools initial design, quality and the accuracy it was built to so it will repeat to those allowable + - one count numbers. #3 Resolution, that one is confused by far too many as a tools accuracy limit, there NOT the same thing. Resolution is just how many digits there are to the right of the decimal point and 100% not the tools built in repeatable accuracy it's actually capable of. For an extreme example, you could have a millionths reading micrometer, yet if it's damaged, highly worn or really poorly made it might only be capable of somewhere within + - .003" accuracy or worse. So when I'm shopping for any new measuring tool I make sure to check it's specifications for it's repeatability and accuracy as my first priority. DRO's are a prime example of what I mean. Machinist squares are a bit different though, generally all you'll see listed are numbers for guaranteed squareness and blade parallelism. But all of the better fixed blade machinist squares will always show those guaranteed numbers. So yes they do list there accuracy. Drop that square just once and those numbers are worthless though. I'll fully admit fixed blade machinist squares can fit into areas that what I use won't. But given what any decent brand name square now costs verses there guaranteed accuracy that I think is fairly poor, I still don't own any. Instead I use pre checked and known to be square within a few 10ths 123, 246 blocks, or a B & S 558 cylindrical square that can actually measure by how much any out of square item is, or a dti preset against a known square. In one way or another I've always been able to work around not having any of those machinist squares.For your table saw blade checks? I'd just use one of my 123 or 246 blocks. There cheaper, probably as accurate as most of the better squares unless you pay really big bucks, and can do other jobs as well. Plus there a lot more durable.

Certified machinist squares with guaranteed low allowable inaccuracy's are going to be expensive no matter where there being made. But for myself I'll no longer buy any metrology related equipment that isn't clearly stated it does come with that guarantee. As one example, I've got a 4" & 6" pair of no name digital calipers I was given for a couple of magazine subscriptions. There incapable of repeating even on expensive gauge blocks within about .010" and being fully certain of the measurement. Sometimes two or three measurements will be almost correct and then they'll throw one out .005"-.010" away from those. So they can't repeat well enough to be trusted and are still full of grinding dust right from the factory. There not even worth pulling apart for a proper cleaning and deburring since the way they eat battery's there worthless to me. There only value is a good reminder to not ever pay real money for anything like them. :-)

There's another way of checking the accuracy of a non adjustable solid machinist square that's fast and dirt simple if you have a mill, a good trustworthy milling vise with a straight properly ground fixed jaw and a 10ths capable dti. Zero the mill vise to the tables X axis as normal. ZERO/ZERO to 10ths on the fixed jaw if you can manage it. Run the Y axis all the way towards you, set the squares short fat end against the mill vise's fixed jaw, and it's thinner long blade pointing towards you. GENTLY close the vise just enough to hold that short leg of the square against that fixed jaw. Use a magnetic base fastened to somewhere else on the mill that's not on the X or Y axis. Zero the indicator tip against the edge of the square leg that's pointing towards you at the end closest to the column, lock the table X axis, then run the table Y axis in so the indicator tip runs along the squares blade. Any number more or less than zero write it down. Loosen the vise and flip the square 180 degrees to the OTHER side of the vise and repeat the exact same test. Any difference in the two numbers is the inaccuracy present on that blade edge of the square. With that you can then use a mike to measure how parallel the blade is to the edge you just measured. If the square is in fact square and the measurements show the blades being parallel, then the inside is true to the outside that you've checked. You could also check the inside edge against the outside with that mill vise and a dti if you set the square in the vise to have enough room to get the ball tip of the dti on both edges and run the same test for each edge.

There's an old carpenters trick I picked up years ago for tuning a square but I sure wouldn't use it on a good machinist square for obvious reasons. I've yet to see any of the last 1/2 dozen carpenters squares I've checked that were within what I'd call square at all Bill. A few I've done were out a lot. But to the level of squareness there needed for, they can be checked and more importantly adjusted to be square fairly easily. Possibly you already know this? But there's probably others here that might not. Find a large sheet of plywood with at least one factory cut edge. That's your reference edge, tape a large piece of paper or thin cardboard as long or longer than the squares long leg just barely under the reference edge and to the face of the plywood. Set the carpenters square with the short leg on that reference edge and use a sharp pencil to draw a line straight down the paper while being careful to keep the pencil at the same angle to the squares edge the full length of the line. Again flip the square 180 degrees so the same short leg is pointing the opposite direction on your ply reference edge. Move the square to where the next line you draw is on top of the first line. Draw another line, any out of squareness will show immediately. If it's out? And almost 100% for sure it will be, figure out in which direction. If the square shows less than a true 90 degrees (the lines move away from each other at the bottom and to the left while drawing the second line) then the squares legs need to be spread apart a bit. You do that by using a hammer and center punch on the INSIDE EDGE of where the two legs of the square meet. If the square shows it's more than 90 degrees, (the lines will widen with the second moving towards the right) then you center punch almost out at the point of where the two legs meet to pull the legs of the square in. This isn't all that intuitive about how hard to hit the center punch or how many marks it might take. It's more trial and error and constant rechecking by drawing more lines until you get it correct. I'd say I can now get a carpenter's square well under 1/64" over 24". The first time I did this took me almost an hr, now I can do one in about 1/2 an hr. And after you do this even once you'll then start to look after even a carpenters square a lot better because it's not hard to get them back out of square. A side issue is most of the stamped steel carpentry squares aren't exactly straight on there edges. That's up to you how you file or stone those edges true and parallel before you begin tuning them for square. And in hindsight, don't ever tell anyone about you know how to do this or you'll get that experience of doing a few more like I have. :-)
User avatar
rmac
Posts: 787
Joined: Tue Jun 26, 2012 12:48 am
Location: Phoenix, Arizona

Re: How precise are measuring instruments?

Post by rmac »

atunguyd wrote: Sat Dec 25, 2021 11:29 pm But if my ruler says that the 10cm block is 10mm then it is accurate.
If your ruler says that the 10 cm block is 10 mm then you have a problem. :D
User avatar
liveaboard
Posts: 1971
Joined: Sun Dec 08, 2013 1:40 pm
Location: southern Portugal
Contact:

Re: How precise are measuring instruments?

Post by liveaboard »

A friend of mine who was a machinist had a favorite joke.
"Measure it with a ruler!"
User avatar
mcostello
Posts: 963
Joined: Sat Jan 04, 2003 10:45 pm
Location: Lancaster, Ohio

Re: How precise are measuring instruments?

Post by mcostello »

Mark it with chalk,
Cut it with a torch,
Grind to fit,
Paint to hide.
Mr Ron
Posts: 2126
Joined: Tue Dec 22, 2009 12:36 pm
Location: Vancleave, Mississippi

Re: How precise are measuring instruments?

Post by Mr Ron »

Thank you Pete for your explanation. Are you saying that if I get a reading of .355", that the piece could actually be .354 or .356.
Mr.Ron from South Mississippi
Post Reply