The best elegant solution to override 65k rows limit in a sheet

H

Harlan Grove

(e-mail address removed) wrote...
....
the point of the matter is CHOOSING THE RIGHT SOLUTION FOR YOUR
ENVIRONMENT. And since all you idiots use is excel-- and you're doing
this on Windows--- it makes perfect sense to use locally installed MSDE
on your desktop.

What about those using Excel on Macs?
I'm not talking about running out and getting hundreds of new unix
servers.
I'm talking about using a FREE--- FUCKING FREE--- database engine that
_IS_ sql server.

MSDE is free.

Now how about business PC users who work for companies that
*EXPLICITLY* prohibit them from installing nonapproved software on
their PCs? Unless their IT department installs MSDE, they can't use it
without putting their jobs at risk.

And would everyone have their own copy of MSDE on each of their
machines with all their own tables only on their own PCs? And how would
this address your stated concern about data islands?
and i can do all your permutations whenever you want. It's called a
cartesian-- and there is no realistic way to do this in excel.. you
HAVE TO use a database to cartesian data.

I already pointed out the foolishness of trying to use a cartesian for
this. See

http://groups.google.com/group/microsoft.public.excel/msg/2c1e73ffd974b4d2?dmode=source&hl=en

(or http://makeashorterlink.com/?I3B12499B ). For permutations of 8
tokens, this requires 16,777,216 cartesian product nodes for 40,320
permutations. BTW, how would one do this *WITHOUT* the OLAP add-in? For
that matter, how would one do it *WITH* the OLAP add-in, i.e., where's
the trivially simple code/instructions to condense the 16,777,216
cartesian product nodes (8^8) into 40,320 permutations of 8 distinct
tokens (8!)?

As for higher order permutations, it may be a pain to do them in Excel
due to the 64K row limit, but it's still possible without unduly
wasting system resources. There are 3,628,800 permutations of 12
distinct tokens (10!). That'd require 56 worksheets if only 10 columns
were used in each worksheet or 3 worksheets if permutations were listed
in columns A-J, L-U, W-AF, etc. in each worksheet. A cartesian product
approach would require 10^10 nodes, which would tax the virtual memory
capacity of most PCs currently in use.

So you could try to do it using cartesian products, but even you would
discover that it's nowhere near simple that way, and you'd run out of
system resources long before a factorial order algorithm in Excel
would.
I dont agree with your 'small amounts of data with lots of calculations
is better in excel'

You disagree?! No! Really?!
For starters
a) you can't always forecast how many records you're going to have
tomorrow, next week, next month, next year.

Which is why most of Excel's functions that take range or array
arguments ignore blanks. Also, it's not that hard to use dynamically
resizable ranges in Excel.

If you know what you're doing in Excel (as you obviously don't), you
can deal pretty easily with changing numbers of data entries. And data
doesn't have to be in record form.
b) Excel ISNT any better than a database for simple math like you
simpletons claim is 'too complex for a db'

Excel and databases provide the *SAME* basic math functionality.
Arithmetic and simple aggregation functions, e.g., SUM, COUNT, AVERAGE,
MIN, MAX, etc. Access even provides many built-in functions similar to
Excel's built-in functions.

However, it's not the functions and operators that hinder dbms's for
math, it's the lack of simple indexing and recursion constructs. Take
another look at the queries *I* provided to generate amortization
tables. You consider them simple compared to the Excel formulas I also
provided in contrast? If you could come up with a simpler database
aproach, PROVE IT and post your instructions.
c) Excel ISNT any EASIER than a database for simple reports. I can run
circles around you, you idiot.

17 years ago I produced a particular report that contained a grand
total of 7 numbers. 4 of those numbers came from a mainframe report to
which the then named MIS department was unwilling to provide online
access (via mainframe terminal emulation and access rights to view
archived print jobs stored on tape). One of the numbers came from the
previous report (and the 123 report template workbook had a \0 macro to
pull it automatically). The last two numbers were formula-generated,
and the formulas never changed (they were perverse proration formulas).

Could this have been completely automated? Yes. However, the MIS
department at the time was unwilling to do so. Could the department in
which I worked have used a database to have automated this? Possibly,
but it would have taken more time and effort to have loaded a dbms and
entered the data into it then generated the report than it did to open
the 123 workbook and let it's \0 macro pull in one number from the
prior report, script user entry of the 4 new entries from the mainframe
report using {?}~ macro statements (typing would be identical in
spreadsheet and dbms), then automatically saving the new report and
printing it.

There's a remote chance a dbms could have created this particular
report as quickly as a spreadsheet could (on the spreadsheet side the
greatest amount of time was spent entering the 4 numbers), but there's
no way a dbms could have produced it more quickly.
normalization isn't possible in excel.. denormalization in excel makes
things IMPOSSIBLE.

Normalization is explicitly unwelcome in Excel. Most of the data
structures uses in spreadsheets look like small scale pivot tables or
crosstabs, which are decidedly not normalized but are much more
efficient data structures in spreadsheets given the way spreadsheets
dereference spreadsheet data.
and you can't even enforce referential integrity.
....

And the lack of mechanisms to enforce referential integrity is what
makes spreadsheets far more flexible than dbms's.

If you view everything as a potential database, normalization and
referential integrity are essential. Referential integrity is also
essential in spreadsheets, but it needs to be enforced by the
user/developed because spreadsheets can't provide mechanisms to enforce
it without sacrificing a considerable part of what makes them more
suitable to particular calculation-intensive tasks than dbms's.
The point of the matter is that ANY DATA THAT YOU HAVE SHOULD BE STORED
IN A DATABASE

Much of it is. Databases do make good data storage subsystems.
spreadsheets are TOTALLY overused throughout the world. any time that
you are making the same report week in and week out.. you should be
using a database.

Agreed. Repetitive tasks in which nothing changes other than data
that's available from central databases are the ideal example of what
databases do well (much better than spreadsheets). You're right about
that. Where you're dead wrong is believing that makes up most of
business PC use.
Excel just ISN'T FUNCTIONAL enough to do jack shit.

Well, not for you certainly since you couldn't tell your fundamental
orifice from a circular reference.
MSDE and SQL Server have taken over the world.

More repeated misinformation. They're still #3.
YES.. I DO CLAIM THE EVERYTHING THAT YOU DO FITS INTO A NICE LITTLE
COOKIE CUTTER. OR MAYBE A HUNDRED DIFFERENT COOKIE CUTTERS.

Ignorance is such bliss! You have no clue, and you're proud of it!
Your inability to adapt to technology means that you should be drinking
wine out of a paper bag on the side of the street.

You really don't understand that most people aren't hired specifically
to write code or build applications.

It might take me a few weeks to figure out how to do your job. You
couldn't figure out mine if you had years to do so.
You are replaceable. . . .

We all are. However, you're interchangeable with a lot of other
database grunts.
 
A

aaron.kempf

you work for a bunch of unix fags anyways so who gives a **** what they
say?
MSDE IS INCLUDED ON THE OFFICE INSTALLATION DISK. YOU CAN TELL THEM TO
**** OFF IF YOU CANT INSTALL IT-- IT IS PART OF OFFICE.
 
H

Harlan Grove

(e-mail address removed) wrote...
....
MSDE IS INCLUDED ON THE OFFICE INSTALLATION DISK. YOU CAN TELL THEM TO
**** OFF IF YOU CANT INSTALL IT-- IT IS PART OF OFFICE.

Is MSDE included on Office Standard installation CDs?

Almost irrelevant since there's a slightly stripped version available
directly from Microsoft's web site,

http://www.microsoft.com/downloads/...d1-a0bc-479f-bafa-e4b278eb9147&displaylang=en

(or http://makeashorterlink.com/?V28F31A9B ).

Which is all irrelevant if you signed an IT policy stating that you
won't install any nonapproved software on the PC your employer
provides. Getting fired for an unapproved MSDE install wouldn't seem to
help anyone do their job.
 
A

aaron.kempf

any IT person that prevents you from installing an OFFICE COMPONENT
should be shot. I swear to god.. JUST WALK AWAY FROM ANY COMPANY THAT
WONT GIVE YOU YOUR OWN OFFICE DISK.

I mean-- you spend; what.. 50% of your time in outlook, excel and word?

If you want to be able to read docs that shipped on the office cd-- you
should be free to do that.. If you want to install an add-in for
excel to be able to XYZ-- that should be your perogative.

i strongly disagree with all these companies that don't give the end
users enough FREEDOM to get their jobs done.

who the hell are they to dictate how you do your job? I just swear to
god.. These idiot IT people run around and won't let you have an Office
CD-- they do a 'minimal installation' -- what happens when you need the
analysis toolpack for Excel?

You call HELPDESK?

OMG i just dont know where the hell these people came up with these
crap ideas.

to be honest harlan; i never realized that there was an Office Standard
Edition.. i honestly thought that all large companies HAD TO USE the
professional edition since Small Business was only for SMB.

But the bottom line.. is that if you work for a company that doesnt
have Access licenses; you can use the Office Developers' edition (the
name changes every version) to create an installer with the Access
Runtime.. this is a FREE option (on the client).. and it's beautiful.
It won't let you write queries-- but it'll let you run forms and
reports-- and that combined with a good access developer; and you've
got all you need (for typical excel dorks you can just let them RUN the
report instead of REWRITING a different copy of it for each week/month)


But the bottom line is that since Office Pro is only $100 more than
Office standard; it's kindof retarded for companies to buy office
standard.

Do you know that Crystal Reports costs?? what, a grand for designing
reports and $50k for having a single CPU report server?

ACCESS is $100 per seat-- tops.. and www.rptsoftware.com has a report
server component for Access that is something like $300 for a SERVER.

It's all about not recreating the wheel. And that's why i dont use
Excel, I dont use Visual Basic (for a bunch of reasons).. Access is a
MUCH better environment. It's faster-- there are no deployment
problems (to speak of).. and it's VERYYYYYYYYYYYYYYYYY powerful.

for quick and dirty apps; I won't use ASP or ASP.net (dont worry i sure
as HELL dont use Visual Studio to do it.. a bunch of Excel dorks and C#
dorks designed that.. for sure)-- if I had a month I can build
amazon.com in ASP--- but that's not the right tool for every project.

The fact of the matter is that, by and large-- you guys are making the
same reports week in and week out.. you make spreadsheet messes-- and
when you hit the limits of Excel; you call in someone to spend a
million dollars and turn it into a crystal report.. or a J2ee
component.. I mean.. WTF?

there is a better way; it is called ACCESS.

I'm just tired of having to copy objects to a hundred different
documents.. I'm tired of cutting and pasting info into Excel... And I'm
tired of having dirty data from spreadsheets infecting my databases.
If Excel was a RELIABLE STORE FOR INFORMATION-- maybe i wouldn't be so
militantly anti-XLS... but as it is; validation needs to be about 1,000
times STRONGER in order to make Excel a decent platform.

And I know that you're tired of this 1-dimensional stuff also. It's
just so 1995 to copy and paste data around like you guys do. It's
error-prone.. it's repetitive.. it's boring..

-Aaron
 
N

Norman Jones

Hi Aaron,

any IT person that prevents you from installing an OFFICE COMPONENT
should be shot. I swear to god.. JUST WALK AWAY FROM ANY COMPANY THAT
WONT GIVE YOU YOUR OWN OFFICE DISK.

I mean-- you spend; what.. 50% of your time in outlook, excel and word?

If you want to be able to read docs that shipped on the office cd-- you
should be free to do that.. If you want to install an add-in for
excel to be able to XYZ-- that should be your perogative.

i strongly disagree with all these companies that don't give the end
users enough FREEDOM to get their jobs done.

who the hell are they to dictate how you do your job? I just swear to
god.. These idiot IT people run around and won't let you have an Office
CD-- they do a 'minimal installation' -- what happens when you need the
analysis toolpack for Excel?

You call HELPDESK?

OMG i just dont know where the hell these people came up with these
crap ideas.

to be honest harlan; i never realized that there was an Office Standard
Edition.. i honestly thought that all large companies HAD TO USE the
professional edition since Small Business was only for SMB.

But the bottom line.. is that if you work for a company that doesnt
have Access licenses; you can use the Office Developers' edition (the
name changes every version) to create an installer with the Access
Runtime.. this is a FREE option (on the client).. and it's beautiful.
It won't let you write queries-- but it'll let you run forms and
reports-- and that combined with a good access developer; and you've
got all you need (for typical excel dorks you can just let them RUN the
report instead of REWRITING a different copy of it for each week/month)


But the bottom line is that since Office Pro is only $100 more than
Office standard; it's kindof retarded for companies to buy office
standard.

Do you know that Crystal Reports costs?? what, a grand for designing
reports and $50k for having a single CPU report server?

ACCESS is $100 per seat-- tops.. and www.rptsoftware.com has a report
server component for Access that is something like $300 for a SERVER.

It's all about not recreating the wheel. And that's why i dont use
Excel, I dont use Visual Basic (for a bunch of reasons).. Access is a
MUCH better environment. It's faster-- there are no deployment
problems (to speak of).. and it's VERYYYYYYYYYYYYYYYYY powerful.

for quick and dirty apps; I won't use ASP or ASP.net (dont worry i sure
as HELL dont use Visual Studio to do it.. a bunch of Excel dorks and C#
dorks designed that.. for sure)-- if I had a month I can build
amazon.com in ASP--- but that's not the right tool for every project.

The fact of the matter is that, by and large-- you guys are making the
same reports week in and week out.. you make spreadsheet messes-- and
when you hit the limits of Excel; you call in someone to spend a
million dollars and turn it into a crystal report.. or a J2ee
component.. I mean.. WTF?

there is a better way; it is called ACCESS.

I'm just tired of having to copy objects to a hundred different
documents.. I'm tired of cutting and pasting info into Excel... And I'm
tired of having dirty data from spreadsheets infecting my databases.
If Excel was a RELIABLE STORE FOR INFORMATION-- maybe i wouldn't be so
militantly anti-XLS... but as it is; validation needs to be about 1,000
times STRONGER in order to make Excel a decent platform.

And I know that you're tired of this 1-dimensional stuff also. It's
just so 1995 to copy and paste data around like you guys do. It's
error-prone.. it's repetitive.. it's boring..

-Aaron

Is it possible that *you* are the same Aaron Kemp who recently posted:

'================================
i just wish that they would take the help for products and make it a
seperate download.. just like books online for sql server.


because i have a dozen problems with their help system per week.


and i work for a company where i cant get an office disk


-aaron
'================================

To refresh your memory, see the full thread at:

http://tinyurl.com/933nc


BTW, how many IT people have you shot in your company?
 
H

Harlan Grove

any IT person that prevents you from installing an OFFICE COMPONENT
should be shot. I swear to god.. JUST WALK AWAY FROM ANY COMPANY THAT
WONT GIVE YOU YOUR OWN OFFICE DISK.

If you don't use MSDE, you should be fired, and if your company won't let
you install MSDE, you should quit!?

You should get a strong dose of reality.
i strongly disagree with all these companies that don't give the end
users enough FREEDOM to get their jobs done.

What you don't understand is that most people's jobs probably could be done,
perhaps more slowly, without any computer at all. Business PCs are just
tools either to simplify users lives OR simplify data entry and control.
Users have some limited say in the former, none in the latter. With regard
to the former, it's their choice as to what works best for them of the tools
their employer provides.
who the hell are they to dictate how you do your job? . . .

Those who pay get to call the shots. You're really unaware of this?!
But the bottom line is that since Office Pro is only $100 more than
Office standard; it's kindof retarded for companies to buy office
standard.
....

Multiply that $100 by the number of seats AND the additional training and
support, and even you should begin to see why some companies stick with
Office Standard.
 
M

MattShoreson

Aaron, it’s such a shame that you devote so much time to denigratin
excel and not your work.

Did you have a bad experience with excel in your childhood?
Did it harm you in any way?

There must be some reason for the hate.

I believe there are professional councillors who may help you out wit
your problems, idiosyncrasies, delusions of grandeur.

Get a life and get back to work
 
A

aaron.kempf

yes, I was assraped by a demon that ate spreadsheets

this demon lives and thrives in each of your heads.. you sit around and
build the same spreadsheet week in and week out

it is not the most efficient way to do business.

you idiots sit around AND RECREATE THE SAME REPORT WEEK IN AND WEEK
OUT.

YOU SIT AROUND AND MAKE INVOICES IN EXCEL.

YOU SIT AROUND AND MAKE COMPENSATION REPORTS IN EXCEL.

YOU SIT AROUND AND MAKE BUDGETING AND FORECASTING APPS IN EXCEL.

YOU GUYS ARE THE BANE OF HUMANITY-- YOU SIT AROUND AND TAKE ADVANTAGE
OF THE DATABASE PEOPLE-- YOUR DISEASED PROGRAM SPAWNS MORE AND MORE
BEANCOUNTERS-- WHEN WHAT YOU REALLY NEED IS MORE AND MORE DATABASE
PEOPLE.
 
A

aaron.kempf

ACCESS DOESNT TAKE TRAINING.

IT TAKES PEOPLE WITH DRIVE TO GO OUT AND LEARN IT ON THEIR OWN.

YOU GUYS ARE SLOTHS; YOU AREN'T ADAPTING TO TECHNOLOGY. YOU GUYS ARE
JERKING OFF WITH SPREADSHEETS AND YOUR FUTURE CONSISTS OF DRINKING OUT
OF A PAPER BAG ON THE SIDE OF THE STREET.

Harlan-- DATA ENTRY DOESNT EVER HAPPEN IN EXCEL YOU IDIOT.

DATA ENTRY HAPPENS WITH A DATABASE. AND YOUR DISEASED COMPANY AND
DISEASED FRIENDS NEED TO WAKE UP AND SMELL THE COFFEE. THERE ARE
BETTER TOOLS FOR BUILDING THE SAME REPORT WEEK IN AND WEEK OUT. THERE
ARE BETTER TOOLS FOR 'DATA ENTRY'.
 
H

Harlan Grove

(e-mail address removed) wrote...
ACCESS DOESNT TAKE TRAINING.

IT TAKES PEOPLE WITH DRIVE TO GO OUT AND LEARN IT ON THEIR OWN.

Any software can be learned by anyone with sufficient wit, sufficient
motivation and sufficient time. It's that third item that explains why
sensible companies spend money on training - the time saved more than
makes up for the training cost FOR A *FEW* PEOPLE. It doesn't make
sense for *ALL* people, and most Excel users aren't going to spend
their weekends coming into the office to learn Access on their own, and
they're not going to waste their weekdays learning it because they have
their REAL JOBS to do.
Harlan-- DATA ENTRY DOESNT EVER HAPPEN IN EXCEL YOU IDIOT.

Data entry happens almost exclusively in spreadsheets and two rather
ancient Clipper apps (admittedly database, but xBase rather than SQL)
where I work. You may know how things are done where you work (unlikely
since you'd thoroughly demonstrated your narrowness of perspective),
but you have no idea how things are done where I work.

I can't call you an idiot. That'd imply there was some slight chance
you could recognize the overwhelming stupidity of what you write.
DATA ENTRY HAPPENS WITH A DATABASE. AND YOUR DISEASED COMPANY AND
DISEASED FRIENDS NEED TO WAKE UP AND SMELL THE COFFEE. THERE ARE
....

Data entry happens in forms, be they database forms, web forms,
spreadsheet forms, whatever. Maybe most data is stored in database.
That's fine. I'm on record as stating that databases are ideal as
storage subsystems. They make decent reporting tools too. They're just
not particularly flexible for analysis. Of couse if you assume several
tens of thousands of dollars of additional software running on top of
some database, that'd help, but most Excel users won't have such
additional software available for their use, so assuming they do would
only futher establish the utter stupidity of your rants.
 
A

aaron.kempf

YOU'RE NOT IMPORTING FROM EXCEL BACK INTO ACCESS YOU MERELY RUN REPORTS
ON THE NUMBERS

THE DATA ISNT GOING FROM EXCEL TO ACCESS TO EXCEL

ITS GOING FROM THE SOURCE-- TO ACCESS-- AND IT CAN BE ******** REPORTED
ON ***************** through a pivot table.

pivot tables aren't an excel phenomenon.

there are a dozen differenent ways to use pivotTables without opening
excel

I would reccomend OFFICE WEB COMPONENTS over excel any day of the
week.. emailing huge spreadsheets around was passe in 1995
 
H

Harlan Grove

(e-mail address removed) wrote...
YOU'RE NOT IMPORTING FROM EXCEL BACK INTO ACCESS YOU MERELY RUN REPORTS
ON THE NUMBERS

If the numbers aren't in one's company's database, e.g., any
information fresh from customers, Access could do squat all with it.

Possibly it could be scanned and sent through OCR. Obviously we'd
differ on the best software to use to check the scanned/OCRed result,
but I'd use Excel or Word EVERY TIME rather than Access in that
situation.
THE DATA ISNT GOING FROM EXCEL TO ACCESS TO EXCEL
....

For me, if original data entry is in Excel, it usually stays there,
never moving on to Access or Oracle or DB2. In the rare situations it
does move on to Access, it never comes back to Excel because it's
already available in Excel, and I find it easier & quicker to pull such
data from other XLS files than from Access.
I would reccomend OFFICE WEB COMPONENTS over excel any day of the
week.. emailing huge spreadsheets around was passe in 1995

And how would one transmit a lovely OWC object to anyone else if the
sender has no rights to create web pages on the web server, which is
the situation of most employees in most companies whose jobs aren't in
web development? Lemme think - convert it into an Excel file!

When are you gonna stop being an idiot?
 
H

hhalle

Try Corel's Quattro Pro 12, The limit is 1000000 rows (A1 to IV 100000)
and Tabs from A through Z

It's an investment but it works.

I use it when running a 175000 records csv file and it works, just keep
an eye on the virtual memory but even this is handled better with
Corel.

Hans
 
J

JayWind

When I try to open a large DBF in either ACCESS or QUATTRO,
I get the error message INDEX DOES NOT EXIST.

I am aware that the values in Column A are not unique.

Any solutions?

-- Thanks Jay
 
G

greaseman

I wonder if this Aaron idiot eats with the same mouth that the rest o
his garbage comes from?

I wonder what rock he crawled out from? Whatever one it was, he shoul
crawl back under it.

Incidentally, what other names does this moronic troll use to snea
into forums and newsgroups
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top