Vlookup nightmare

A

aaron.kempf

so you expect me to go through my Excel file, cell by cell.. and set 50
fields out of my column of 10k records-- you expect me to set these
fields to numeric by hand?

HAHAHA lol

and no, i dont do excel macros-- well i do them; but i choose to use my
VBA in the correct tier; and importing spreadsheets into a database
shoudn't require me to shape the data in an unpredictable, unscalable
manner.

i just want an answer.. and when i choose text-- this should preserve
EVERYTHING in a cell and display it without a format mask.

it just drives me crazy that excel is so popular; im tired of choking
on XLS data.
 
A

aaron.kempf

but SERIOUSLY

thank you for finally listening to me harlan i just go crazy when i
can't find an easy answer.

and i've had to import THOUSANDS of spreadsheets in my day.. i'm just
tired of the bugginess.

I saw some new article about VSTO 2005 making this easier/more
reliable.. i'll have to check that out
 
H

Harlan Grove

(e-mail address removed) wrote...
so you expect me to go through my Excel file, cell by cell.. and set 50
fields out of my column of 10k records-- you expect me to set these
fields to numeric by hand?

No, I expect you to select entire columns that need to be treated as
text then run Data > Text to Columns. If you have 50 such
columns/fields, then yes I do expect you to do this 50 times. The
intelligent alternative is formatting these columns as text BEFORE you
enter anything in them.
and no, i dont do excel macros-- well i do them; but i choose to use my
VBA in the correct tier; and importing spreadsheets into a database
shoudn't require me to shape the data in an unpredictable, unscalable
manner.

You're the one entering numbers as numbers into columns that should be
treated as text. But how would a macro to fix this be unpredictable or
unscalable? All you'd need to convert EVERY number in ANY cell in a
column containing other text entries besides the field name would be
something like this.


Sub FixAaronPlease()
Dim ws As Worksheet, c As Range, r As Range

For Each ws In ActiveWorkbook.Worksheets

For Each c In ws.UsedRange.Columns

If Application.CountA(c) - Application.Count(c) >= 2 Then

For Each r In c.Cells

If VarType(r.Value2) = vbDouble Then _
r.Value = Format(r.Value2, "\'0")

Next r

End If

Next c

Next ws

End Sub


This should be simple enough even you could figure out the logic. All
you have to do is have the problem workbook active before running this.
i just want an answer.. and when i choose text-- this should preserve
EVERYTHING in a cell and display it without a format mask.
....

It does AS LONG AS you format as Text FIRST, then enter data.
 
H

Harlan Grove

(e-mail address removed) wrote...
thank you for finally listening to me harlan i just go crazy when i
can't find an easy answer.
....

You're welcome.

The main point is FORMAT BEFORE DATA ENTRY. This is necessary if any
entry would consist of 16 or more decimal numerals.
 
A

aaron.kempf

ok well this XLS is a dump from SAP
so uh.. do i really have to open it in excel and do this crap in my
ETL?

its not like people are entering records into Excel-- lol-- 2 people
can't even use the same XLS at the same time i mean-- i'd rather use
text files than Excel for data entry

-aaron
 
A

aaron.kempf

wait i get it i think.. i'll look into that text-to-columns things
thanks harlan.. its' nice that a week into this thread we actually
start talking to each other like grownups lol

thanks
 
H

Harlan Grove

(e-mail address removed) wrote...
ok well this XLS is a dump from SAP
so uh.. do i really have to open it in excel and do this crap in my
ETL?

'ETL'?

If you try importing tables in an XLS file generated by SAP directly
into any database I can thing of, if the database sees any text in any
column/field, it'll make set that column/field's type to text. If there
are numbers appearing in scientific notation, you're screwed.

AFAIK, nothing but Excel could fix this. So, YES, you do need to
preprocess all these XLS files in Excel to convert numbers into text in
normal number format in the columns that any database would import as
text. See the macro I already provided in another response. It'd be
easy enough to adapt it to work under Automation.
its not like people are entering records into Excel-- lol-- 2 people
can't even use the same XLS at the same time i mean-- i'd rather use
text files than Excel for data entry

If these XLS files are generated by SAP, then no human's entering
anything. Also if so, then you should try to get whoever generates
these files to export them in some other format. DBF (admittedly no
prize either) would be an improvement over XLS. CSV may not work if the
numbers in text fields would be written in scientific notation.
 
A

aaron.kempf

extract, transform, load

db-speak for moving data around

yeah i wish i could have gotten some format other than excel.. but that
is what they knew how to do; and they had a couple other systems
(excel) that used the data in this format.

it's ok-- i just figured that there should be an easy answer..

I _SHOULD_ be able to tell excel to 'never, ever, ever, ever, use
scientific notation'
 
H

Harlan Grove

(e-mail address removed) wrote...
....
I _SHOULD_ be able to tell excel to 'never, ever, ever, ever, use
scientific notation'

Final comments.

This isn't possible. Excel isn't alone in using a 'general' format to
represent numbers under a given magnitude in 'normal' format but
numbers of greater magnitude in scientific notation. Anything using C's
printf with %[..]g format, any spreadsheet, even Access itself (!!!!)
with no format specified for Double type number fields will represent
high magnitude numbers in scientific notation. If your XLS files are
being generated by SAP, then it's a near certainty ALL cells have
general number format.

I'll repeat a point I made before. It's the database's fault that it's
importing each cell's .Text rather than it's .Value property. Maybe
importing the .Text property would seem to make sense for text fields,
but it's REALLY DUMB. Indeed, Access itself imports the .Value property
of cells containing text, but the .Text property of cells containing
numbers. You can test this using the Excel custom number format

"foo-"0"-bar";;;"foo-"@"-bar"

Given the table (cell contents shown)

Field
1
2
a
b

Excel will display this as

Field
foo-1-bar
foo-2-bar
foo-a-bar
foo-b-bar

but Access will import it as

Field
foo-1-bar
foo-2-bar
a
b


I just don't see this as Excel's fault. IMO, either Access is fubar, or
the ODBC module for XLS files is fubar, but Excel itself isn't to
blame.

FWLIW, OpenOffice beta 2.0 Base imports the .Text property
consistently, so the Excel table above comes in to Base as

Field
foo-1-bar
foo-2-bar
foo-a-bar
foo-b-bar

Further, Microsoft Query pulls the numbers AS NUMBERS and pulls the
text as NULLs, no mater how the records are ordered, so it seems Query
decides a field is numeric if there are any numeric entries in it while
Access decides a field is text if there are any text entries in it.
Long ago I gave up expecting any design consistency between different
Microsoft programs, so this doesn't really bother me. However, the fact
that Query pulls the numbers as 1 and 2 shows that it's not Excel's
ODBC module to blame for passing .Text properties of number entries in
text columns. It's pretty clear Access is to blame for your import
problems.

When are you going to stop using such buggy software as Access?!
 
A

aaron.kempf

hahaha lol
it's not an Access problem.
i just want to be able to shut off scientific notation. I NEVER EVER
EVER EVER want that bastard of a program to try to use Scientific
again.

i like your 'its not a problem with excel; its a problem with excel
odbc'--

thats cute bud, you should work at microsoft-- you could pass the buck
with the best of those passbuckers

this is the field that excel is screwing up
Field
foo-1-bar
foo-2-bar
9.3434345345e10
foo-a-bar
foo-b-bar
foo-1-bar
foo-2-bar
9.3434345345e10
foo-a-bar
foo-b-bar
foo-1-bar
foo-2-bar
9.3434345345e10
foo-a-bar
foo-b-bar
foo-1-bar
foo-2-bar
9.3434345345e10
foo-a-bar
foo-b-bar
foo-1-bar
foo-2-bar
9.3434345345e10
foo-a-bar
foo-b-bar
foo-1-bar
foo-2-bar
9.3434345345e10
foo-a-bar
foo-b-bar


I just realy think that excel should have these abilities:

a) to turn off all '.text' and only display '.value'-- then you could
SEE what is wrong with the DATA not excels little interpretation of the
data.

I dont want a difference between .text and .value

that is what would make excel useable for me.
take away .text entirely

and I dont know if i believe your little example there.. i just dont
know if i believe you.

either way; microsoft isn't taking office seriously; and i can't WAIT
for the day that there are viable alternatives to Access in the
marketplace.

as it is; they have a monopoly-- Excel and Access are each their own
monopoly.. and Microsoft isn't motivated to change this because they
have a long-standing practice of abusing Access developers.

I mean-- how many bugs do you have to see that say 'this is a known
issue' before you get outraged?

I'm just outraged that a simple operation-- moving data between excel
and access is soooo difficult.

and i'd rather that THEY FUCKING FIX IT rather than try to shove XML
down our throats.
 
H

Harlan Grove

(e-mail address removed) wrote...
it's not an Access problem.

How d'ya figure that since the numbers are still numbers when the XLS
file is loaded in Excel, but become text representations of scientific
notation when imported into Access?
i just want to be able to shut off scientific notation. I NEVER EVER
EVER EVER want that bastard of a program to try to use Scientific
again.

Too bad. It's not going to be rewritten to suit your whim. Gripe all
you want, but you'll have to live with it if you need to process XLS
files.
i like your 'its not a problem with excel; its a problem with excel
odbc'--

thats cute bud, you should work at microsoft-- you could pass the buck
with the best of those passbuckers

As I proved later in the same message, Microsoft Query uses Excel ODBC,
and it has no trouble importing numbers displaying in scientific
notation in Excel as numbers in 'normal' format. To me, that proves
it's not Excel ODBC that's to blame. As repeated above, it's not Excel
itself either that's to blame.

That leaves what?

....
I just realy think that excel should have these abilities:

a) to turn off all '.text' and only display '.value'-- then you could
SEE what is wrong with the DATA not excels little interpretation of the
data.

Easy enough to display in XL. Tools > Options, View tab, check
Formulas.

To repeat, it's not Excel that's forcing Access to pull the .Text
rather than .Value property of number entries in columns Access decides
to treat as text, it's Access that's doing this.

This is a bug in *ACCESS* that should be fixed. It should ALWAYS import
the .Value property, and if that's a number destined for a text field,
it should pass it through something like VB's CStr.
I dont want a difference between .text and .value

There's a reason Excel provides both properties, but the usefulness of
..Text is limited to Excel. Access shouldn't be using it, but (to
repeat) THAT'S A BUG IN *ACCESS*!
that is what would make excel useable for me.
take away .text entirely

Too bad for you because .Text is useful to Excel developers. It's NOT
going away, so get used to it.
and I dont know if i believe your little example there.. i just dont
know if i believe you.

I gave the steps I followed. Perform them yourself and see the results
yourself. The only things preventing you from verifying what I wrote
are your own laziness and/or your inability to follow those steps.
either way; microsoft isn't taking office seriously; and i can't WAIT
for the day that there are viable alternatives to Access in the
marketplace.

There is. Paradox in WordPerfect Office. Damn shame it's not sold
stand-alone. Otherwise you'll have a LONG wait for OpenOffice Base to
become useful or capable.
as it is; they have a monopoly-- Excel and Access are each their own
monopoly.. and Microsoft isn't motivated to change this because they
have a long-standing practice of abusing Access developers.

They have a long-standing practice of doing as little as they can get
away with and call it a major version upgrade then sell it for $$$. I
don't use Access enough to know the changes from 97 to 2002 (or 2003),
but Excel almost nothing in Excel has changed since 2000.
I mean-- how many bugs do you have to see that say 'this is a known
issue' before you get outraged?

Microsoft has a VERY THICK HIDE. FWIW, they can't even get Excel's MOD
function right for moderately large (well within the long integer
domain) numbers even though all they'd need to do is call either their
own standard C library's fmod function or the FPU's FPREM1 operation.
I'm just outraged that a simple operation-- moving data between excel
and access is soooo difficult.
....

Agreed. Blame Access.
 
A

aaron.kempf

i mean seriously.

Excel is just overused; it is a disease.

and the company that sells it is a worthless, headless, monster.

they sit around and say 'it's a known issue' rather than FIXING BUGS
and i am starting a class-action lawsuit against microsoft, regarding
bugs in Office.. the website will be coming soon
 
A

aaron.kempf

i wont blame Access

I blame Microsoft-- the worlds richest software company can't make a
decent product.. i just wish that they were broken up 4 years ago in
the antitrust proceedings.

they're obviously incompetent

and they're all excel dorks so **** microsoft

I dont want to take away .TEXT
i just want to be able to toggle it on and off-- and i would choose to
ALWAYS toggle it off

I dont want frilly BS

i dont need your little formatting crap-- all it does it make my life
harder

all i know is that the number comes in as 92348798489734 and Excel
makes it pretty and decides that i want scientific notation-- i mean--
WHO THE HELL WANTS SCIENTIFIC NOTATION IN EXCEL?

make it the exception; not the norm



ps check _THIS_ out i mean wow-- why didnt this come around YEARS ago?
http://www.devx.com/dotnet/Article/28468

Build this IF Function Parser to Simplify Debugging Excel Formulas
If you're tired of trying to decipher long Excel IF formulas displayed
in a single line of code, try this convenient Formula Formatter add-in
that displays such formulas neatly indented in a popup window.

by Tade Oyebode


I MEAN WOW-- SOMEONE ELSE IS FED UP WITH USING EXCEL IN THE REAL WORLD
 
H

Harlan Grove

(e-mail address removed) wrote...
Excel is just overused; it is a disease.

It may be overused, but that doesn't mean it isn't useful when it's
appropriate. You don't get that.
and the company that sells it is a worthless, headless, monster.

Oh, no! They have a head. Actually several. Microsoft may be unpleasant
and ruthless, cynical and expedient, but they are most definitely NOT
headless. They know they can sell crap because they know there are LOTS
of people who'll pay big bucks for crap.
they sit around and say 'it's a known issue' rather than FIXING BUGS
and i am starting a class-action lawsuit against microsoft, regarding
bugs in Office.. the website will be coming soon

Pointless (so characteristic for you). Do you understand the EULA? Do
you understand US tort or contract law?
 
H

Harlan Grove

(e-mail address removed) wrote...
....
all i know is that the number comes in as 92348798489734 and Excel
makes it pretty and decides that i want scientific notation-- i mean--
WHO THE HELL WANTS SCIENTIFIC NOTATION IN EXCEL?

It'd be hard to find what was written at the time, but general number
format was an innovation in the 1970s. It wouldn't have come into being
if there hadn't been a perceived need.

Dunno about VisiCalc, but 123 defaulted to general number format, and
all spreadsheets have since. 123 also provided tools for converting WKS
files to DBF files, and there were plain and unambiguous rules for what
needed to be done in the WKS file for that to work, and formatting was
top of the list. That's where & when I developed the habits that allow
me to avoid the headaches you're having.

As I've already mentioned, Access also displays very large numbers in
scientific notation when no number format is specified. It's STANDARD
behaviour in most software that can perform arithmetic on floating
point numbers. Get used to it.
ps check _THIS_ out i mean wow-- why didnt this come around YEARS ago?
http://www.devx.com/dotnet/Article/28468
....

Old news. A few people have done similar things to edit long formulas.

Why Microsoft can't provide such a tool isn't really a mystery. It
won't help them sell more copies of Office, and their programmers are
unlikely to use Excel to any great extent, so they haven't developed
anything like this for their own use.
I MEAN WOW-- SOMEONE ELSE IS FED UP WITH USING EXCEL IN THE REAL WORLD

Read more of my posts in other threads. I'm not exactly shy about
pointing out where Excel stinks. But, while there's ample room for
improvement, it's still useful for many tasks.
 
A

aaron.kempf

screw the EULA; I will break it in court.

Microsoft is bending customers over and stealing their money and
putting out crap products.

I mean-- Excel is totally unuseable for anything more complex that a
simple purchase order.. lol
it just was designed my dorks.

Microsoft is headless-- they have never demonstrated any LEADSERSHIP
with Office. All they do is copy other products and they do a crap job
of it.

I ask for Microsoft to start taking Access seriously; and to start
treating Excel like the disease that it is.
 
H

Harlan Grove

(e-mail address removed) wrote...
screw the EULA; I will break it in court.
....

No you won't because the disclaimers in Microsoft's EULA are
essentially the same as the disclaimers in most software licenses used
by companies selling software in the US for the past several decades.
Do you really believe they haven't already been tested in court?!

If you really want to bash your head against a wall over & over, go
ahead. Unlikely anyone here would stop you (or support you).
 
A

aaron.kempf

i dont care if they've been tested in court.

Microsoft is screwing customers. And I'm sick and fucking tired of it.

START FIXING YOUR BUGS MICROSOFT AND STOP TRYING TO SELL US 3 NEW
VERSIONS OF OFFICE IN 3 YEARS

I mean-- seriously here Microsoft-- a bunch of drunk and blind
preschoolers could make a spreadsheet program that is easier to get
data in and out of.

How about you make something that LOOKS JUST LIKE ACCESS DATA PROJECTS
but it is a piece of Excel.

So everything that these monkeys are creating in hundreds of different
spreadsheets-- we can digest and spit out with a real reporting
platform.

Excel can't even print fixed-width stuff correctly.

I mean-- Excel needs reporting abilities.
Access needs more Excel abilities-- like grabbing a range and you see a
subtotal.
And Excel needs to be a format that is easy to get data out of.

I dont think that XML is the answer to everything.. I mean-- you guys
are on crack.

Fix what you are already selling before you try to shove a new
technology down our throats.
 
H

Harlan Grove

(e-mail address removed) wrote...
i dont care if they've been tested in court.

Ah, the strength of your moral outrage will overcome all obstacles.

Grow up!
Microsoft is screwing customers. And I'm sick and fucking tired of it.

Are they the first company to be accused of doing so?!

Grow up!!
START FIXING YOUR BUGS MICROSOFT AND STOP TRYING TO SELL US 3 NEW
VERSIONS OF OFFICE IN 3 YEARS

It's 3 versions in 4 years if you mean 2000 through 2003.

As I wrote before, Microsoft's goal is MONEY, not accolades for their
products. As long as crap sells, Microsoft will sell crap.
I mean-- seriously here Microsoft-- a bunch of drunk and blind
preschoolers could make a spreadsheet program that is easier to get
data in and out of.

Not cynical enough! Their programmers are quite capable of fixing most
of what's wrong in Excel or Access or the rest of Office. Their
managers, on the other hand, don't want to invest in such unprofitable
areas as improving software that sells quite well as-is. There's no ROI
on fixing anything in Office, so they're far more likely to add
blinking text to Excel and Access than to fix anything or (dare I even
imagine it!) add functionality. [Excel hasn't had a new worksheet
function added since Excel 97 or augmented since 2000. A few have been
fixed in 2002 and 2003, but nothing has been added.]
How about you make something that LOOKS JUST LIKE ACCESS DATA PROJECTS
but it is a piece of Excel.

You need to realize something. Queries built with Access's Query
Builder don't provide more than a fraction of what Access supports in
SQL queries, but most spreadsheet users don't know SQL (and you're
deluded if you believe it's easy for most people to learn quickly).

That said, it would indeed be nice to have a high-powered query
facility built directly into Excel *both* as a menu command (replacing
Microsoft Query, which is very underpowered) and as a worksheet
function (replacing SQL.REQUEST).

For that matter, it'd also be nice to have Edit > Find, Edit > Replace,
SEARCH, MATCH, {H|V}LOOKUP and SUBSTITUTE accept VBScript-like regular
expressions (like OpenOffice Calc accepts a somewhat weaker set of
regular expressions).

It'd be nice for LEFT, MID, RIGHT, REPLACE, SUBSTITUTE, FIND and SEARCH
to interpret negative positional arguments as right-to-left.

It'd be nice if Excel's object model included 3D references to make it
possible to pass arguments like first:last!A1:X99 to udfs.

It'd be nice to add a rendering layer to Excel to process standard HTML
formatting tags in cells' .Text properties, e.g., so formulas returning
something like

your result is: <B>100</B>

and display the portion outside the tags in the cell's given format but
the portion inside the tags in boldface.

Pipe dreams!!!
So everything that these monkeys are creating in hundreds of different
spreadsheets-- we can digest and spit out with a real reporting
platform.

Only hundreds?
Excel can't even print fixed-width stuff correctly.

Yes it can if you use monospace typefaces.
I mean-- Excel needs reporting abilities.

Here I disagree. Excel isn't well suited to generating canned reports.
Adding reporting features would make worksheet design more complicated,
while if reporting were generated outside of worksheets, few Excel
users would bother using it.
Access needs more Excel abilities-- like grabbing a range and you see a
subtotal.
And Excel needs to be a format that is easy to get data out of.

XML!

You may hate it, but it *IS* general and more open than anything else
anyone has come up with. And before you mention any dbms format, binary
images of dbms tables aren't standardized, nor are binary file formats
all that easy to deal with if they can't just be read using C-like
fscanf record parsing.
I dont think that XML is the answer to everything.. I mean-- you guys
are on crack.
....

So what else? XML is at least plain text, so no screwing around with
binary files or binary representations of numbers. And dealing with
hierarchies isn't all that difficult (though it is unlike anything
you'd be used in rdbms's).
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top