In my years of working with computers, I have found that there are always really odd things that happen. Sometimes processes simply stop working or don’t complete. Other times, things appear to be working when they are not.
While there can be hardware issues, these have been really rare. But still, weird things happen for no apparent reason.
I am wondering if there is an underlying assumption by software programmers that there will always be enough memory to handle any process. Is there a chance that adding more RAM to the system will stop a lot of oddities from occurring in the first place? Is there any evidence that such is the case?
(One would think that such a thing is not a big issue in this day and age with large RAM capabilities, but one never knows for sure.)
I read things in the forum here that are clearly a misconfiguration, but in other cases the story is really strange.
My experience has been that lack of memory can usually be diagnosed from error messages, performance dropping etc. There is usually a way to see how much free RAM is available too.
You are correct though in that programmers can assume that RAM and Storage space will always be available, you only have to look at the bloat of eg Windows over the years.
OOM failures are very common in hosting and here because people like to run on really small and cheap virtual machines. Without care, running a full stack of hosting apps on a very small VM is asking for trouble.
Of course programmers do not assume infinite memory, but programmers cannot know what memory will be available or what else will be competing for memory. Everything in the stack allocates memory via standard mechanisms, which the OS gives it. The problem is that when memory is exhausted there may be no safe way to continue. Programs cannot operate without memory allocations working. There are programs in critical environments that use a fixed allocation of memory and never release memory or allocate any more…but, this is an inefficient use of memory. Most programs memory usage comes and goes based on what it’s doing, and there’s no reason to hold on to the most memory it’ll ever need forever.
In a system like that, you can’t run close to the edge. At some point you’ll have a bunch of mail come in triggering the scanning stack, a bunch of web requests triggering database activity and PHP or other language allocations, and a backup process running, and something has to give.
TANSTAAFL. You can’t have perfect reliability and perfect efficiency and perfect performance in the same program on modest hardware. If you want NASA level reliability, you have to spend NASA kind of money on the hardware and make the kinds of compromises NASA makes in their software and accept the costs.
And, yes, we often ask folks to check for OOM errors in the case of weird issues, because it is a leading cause of weird issues. I’m tempted to add a check for it, maybe in the validation page.
Note we also offer to create a swapfile during installation on very small systems, which can mitigate some of the risk of an OOM event. We don’t make it a huge swap (something like 1-2GB, which maybe roughly doubles the memory on a very small system), both because a system that runs constantly from swap would be unusably slow and because we can’t predict how precious disk space is. Usually tiny VMs also have relatively tiny storage, that’s one of the ways low-cost VM providers make VMs not consume too much of their resources. Limiting their size in all dimensions restricts the kind of work one can do on it. Can’t do massive data operations without massive data to operate on.
Once again, TANSTAAFL. A cheap hosting VM is going to be small. You can’t have reliability from a huge stack of applications if you also want cheap. Most of the memory usage in a Virtualmin system is out of our control. We try, and I’m sure most of the programmers in the rest of our stack try, to keep memory usage bounded and only as large as necessary. But, people also like features. Features cost memory.
Yes, I think you all are correct. RAM can be a major possibility for many issues. And sometimes the logs will not be able to report a problem, if the system hangs.
Sometimes we just cheap out. (And not because we are really cheap, but because we can’t afford it. I know I can’t afford government-level hardware.)
Any yes, Joe, every programmer is not the same; it was just a glittering generality. And for sure, there ain’t no such thing as a free lunch (TANSTAAFL).
In any case, memory issues sure may appear to be the consensus for a lot of the problems we see reported. Along with bad configuration and simply not following installation directions.
Another reason for “weirdness” is tinkering. I have had associates that were unable to install or maintain a stable system because they keep wanting to “tweak” this or that. That said, I’ve had to create “fix dovecot” scripts over the years (still have one of those) and a “check that apache is running and start if not” script STILL in place…
Yes, I have noticed a lot of that here in the forum. I sometimes scratch my head and wonder what some folks are thinking. We might use a product like Virtualmin, but then don’t seem to trust the experts who set it up to do certain things in a certain manner. There had to be a good reason, after all.
Another issue you see a lot, is using an OS that is not designed to be a server. Some distros are obviously better for that purpose than others. I’m certainly not going to use Ubuntu for that, or even my every day Debian desktop OS.
Hey KitchM, thank you for not taking what I said the wrong way – I needed sleep… : )
What I wish for is a “Virtualmin for Alpine Linux” but I understand the guys are unable to support linux distros that don’t have a base of paying customers to support it… just would be nice to “tinker” with that… : )
I believe that Virtualmin is created to work on any distro that is actually made to be more of a server and which complies with certain industry standards. As such, it should work well out of the box for anyone. I realize that the interface is fairly non-intuitive, but once a person gets past the learning curve, it seems to do the job quite well.
You know, I’ve planned for 10 years to make “intro to Virtualmin” videos… but… I’m so lazy. I feel like a suite of videos would open the eyes of so many people to the wonderful world of Virtualmin… : )
A good video is always useful. But there are caveats:
To many how-to videos use too high of resolution. I have a 50" monitor and cannot discern the details at all, making them worthless. One should use nothing more that 1024x768, or even 800x600 when creating the vid. Enlarging the cursor is also good.
The next problem is people who mumble or have a thick accent or don’t use proper English. This also can make understanding impossible.
And of course there is the problem with missing steps or not making sure there are no disconnects in the presentation. The whole reason one must have instruction is because the software is not intuitive by nature in the first place. Not filling in these gaps also makes a video useless.
I applaud and support anyone’s efforts to help in this regard. And, no, you’re not lazy. You’re just busy like all of us. And don’t forget that lack of energy we sometimes experience. Such is life, Good Buddy. C’est la vie.
As there are many variations of English which do you class as proper English? Taking accents into account, I do find north american accents difficult to folow but Australian, Indian & European ‘English’ accents seem to be easier to follow, as they appear to use the same sentence and word construction North Americans seem to produce something different so which accent/version of English would you find correct?
that’s your view but people from other places may think otherwise, but RP would be a good place to start as it is the base of the English language, but seldom used these days.
Yeah, that’s right. I like that. At least I understand the word. And I know the speaker doesn’t mean stainless steel. Plain, clear speech makes all the difference.
a.k.a POSH southerners.
